Encrypting files in the browser before a 5 TiB multipart upload to R2
I built pigeon.zip, a free file transfer with no size limit and no account. The server must never see a plaintext byte, and a single file can be up to 5 TiB. These are the design choices that made it work. The
I built pigeon.zip, a free file transfer with no size limit and no account. The server must never see a plaintext byte, and a single file can be up to 5 TiB. These are the design choices that made it work.
The key lives in the URL fragment
Each transfer gets a random key in the browser. The share link looks like https://pigeon.zip/<slug>#<key>. Browsers never send the part after # to a server, so the key never appears in requests, logs or analytics. Whoever has the full link can decrypt. The server stores only ciphertext and an encrypted file name.
Format: gzip, then 1 MiB records
The browser gzips the file and splits it into 1 MiB records. Each record is:
AES-256-CTR(encKey, counter = prefix || u64be(i * 65536), plaintext)
|| HMAC-SHA256(macKey, header || u64be(i) || u8(final) || ciphertext)
The record index and a final flag are inside the MAC, so records cannot be reordered, dropped or truncated without detection.
Why CTR + HMAC and not AES-GCM? The terminal client must produce the same format with only openssl enc and the Python standard library. The encryption key is derived so that it equals openssl's -pbkdf2 -iter 1 key for a derived password. The CLI passes that password on stdin, so no key ever shows in ps.
Uploads go straight to R2
The server only signs URLs. The browser sends presigned multipart PUTs directly to Cloudflare R2, so the app server never streams file bytes, and R2 has no egress fees. That is why the service can be free.
R2 allows at most 10,000 parts and a part size between 5 MiB and 5 GiB. The part size starts at 5 MiB for small files and grows with the file:
const minPartForCount = Math.ceil(fileSize / 10_000 / MiB) * MiB
size = Math.max(size, minPartForCount)
So a 5 TiB file still fits in 10,000 parts. An interrupted upload resumes after a reload.
Links clean themselves up
Each link lasts 21 days or 144 downloads, whichever comes first. A lifecycle rule on the bucket deletes objects after 22 days and aborts stale multipart uploads after 7 days, so storage cannot grow without limit even if the app fails to clean up.
From the terminal
curl -s https://pigeon.zip/api/cli/up/yourfile | sh
It returns a short script that encrypts and uploads with the same format, and prints the link. Read the script before you run it: drop the | sh.
Try it with your biggest file: pigeon.zip. I would like feedback on the crypto design and on very large uploads.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.