Dev.to WebDev 🛠 Dev 👁 0 📖 4 min read

Why Base64 is a third bigger, and why btoa breaks on accented text

Base64 turns bytes into text so they can travel through places that only accept text: JSON bodies, data URLs, email, a CSS file. The price is size. Every 3 bytes become 4 characters, so the output is a third bigger, plus

Base64 turns bytes into text so they can travel through places that only accept text: JSON bodies, data URLs, email, a CSS file. The price is size. Every 3 bytes become 4 characters, so the output is a third bigger, plus padding at the end.

const len = (n) => 4 * Math.ceil(n / 3)
console.log(len(1), len(2), len(3), len(100), len(1000)) // 4 4 4 136 1336

Measured on random bytes from crypto.randomBytes:

bytes     base64 chars   ratio
1         4              4.0000
2         4              2.0000
3         4              1.3333
100       136            1.3600
1000      1336           1.3360
1048576   1398104        1.3333

Small inputs look worse because of padding. A 1 MB file ends up as 1,398,104 characters, which is the number to keep in mind when an API caps the request body. A 5 MB file becomes 6.67 MB:

const n = 5 * 1024 * 1024
console.log(n, 4 * Math.ceil(n / 3), (4 * Math.ceil(n / 3) / 1048576).toFixed(2))
// 5242880 6990508 6.67

That is why the Base64 tool on Toolsy Hub stops encoding files at 5 MB: the encoded text has to sit in memory and on the clipboard, and it is a third larger than the file.

Padding

The = signs fill the last group of 4. One leftover byte gives ==, two give =, three give none:

for (const s of ['a', 'ab', 'abc']) console.log(JSON.stringify(s), btoa(s))
// "a" YQ==
// "ab" YWI=
// "abc" YWJj

Browsers are forgiving about missing padding in atob (atob('YQ') returns a), but a length that leaves a single stray character is an error:

atob('Y') // InvalidCharacterError: The string to be decoded is not correctly encoded.

A decoder that adds the padding back before calling atob accepts both forms. The tool does this, and it also drops a data:...;base64, prefix and ignores whitespace, because that is how Base64 usually arrives when you copy it from somewhere.

The btoa trap with non-ASCII text

btoa takes a "binary string" where every character is one byte. Text outside Latin-1 throws:

btoa('héllo €') // InvalidCharacterError: Invalid character

The sneaky case is text that is inside Latin-1. btoa('héllo') does not throw, but it encodes the single byte 0xE9 for the é, not the two UTF-8 bytes. Anything that decodes it as UTF-8 will not see the same text:

btoa('héllo')                                                  // aOlsbG8=
btoa(String.fromCharCode(...new TextEncoder().encode('héllo'))) // aMOpbGxv
Buffer.from('héllo').toString('base64')                        // aMOpbGxv

The last two match, and the first does not. Encode to UTF-8 bytes first, then turn the bytes into a binary string, then call btoa. Decoding goes the other way with TextDecoder:

const enc = (s) => btoa(String.fromCharCode(...new TextEncoder().encode(s)))
const text = new TextDecoder().decode(Uint8Array.from(atob(enc('héllo €')), (c) => c.charCodeAt(0)))
console.log(enc('héllo €'), text) // aMOpbGxvIOKCrA== héllo €

Do not spread a big array

The one-liner String.fromCharCode(...bytes) works for text but not for files. Spreading 200,000 bytes into arguments overflows the stack:

String.fromCharCode(...new Uint8Array(200000))
// RangeError: Maximum call stack size exceeded

Build the binary string in chunks instead. The tool uses 0x8000 bytes at a time:

let bin = ''
for (let i = 0; i < bytes.length; i += 0x8000) {
  bin += String.fromCharCode(...bytes.subarray(i, i + 0x8000))
}
btoa(bin)

A 1 MB array goes through this in one pass and comes out at 1,398,104 characters.

URL-safe Base64

Standard Base64 uses + and /, which mean something in URLs. The URL-safe variant swaps them for - and _ and usually drops the padding. The three bytes fb ff fe show the difference:

const std = btoa(String.fromCharCode(0xfb, 0xff, 0xfe))
console.log(std, std.replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, ''))
// +//+ -__-

On the way back, swap the characters again and re-pad. JWTs use the URL-safe form, which is why pasting one into a plain decoder sometimes fails.

Guessing the file type from the text

Because the first bytes of a file are fixed, the first characters of its Base64 are fixed too. A PDF starts with %PDF-, a PNG with 89 50 4e 47 0d 0a 1a 0a, a JPEG with ff d8 ff:

console.log(btoa(String.fromCharCode(0x25, 0x50, 0x44, 0x46, 0x2d)))                           // JVBERi0=
console.log(btoa(String.fromCharCode(0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a)))         // iVBORw0KGgo=
console.log(btoa(String.fromCharCode(0xff, 0xd8, 0xff)))                                       // /9j/

So JVBERi0 means PDF, iVBORw0KGgo means PNG and /9j/ means JPEG. The tool checks these signatures after decoding. A decoded PDF can be plain ASCII, so it checks the file type before it tries UTF-8 text. If the bytes match a known type, or are not valid UTF-8, you get a download button instead of garbled text.

To try any of this without writing code, the encoder and decoder is at https://www.toolsy-hub.com/tools/base64. It runs in the browser tab, handles UTF-8 text and files, and decodes both standard and URL-safe input.

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.