Why Non-ASCII PDF Form Values Render as Boxes or Crash Pipelines: WinAnsi vs. Type0 Fonts

A customer types a name like Łukasz, Doamnă, or 田中太郎 into a PDF onboarding form. Your script writes that value to the AcroForm text field and calls save, and the job crashes with Error: WinAnsi cannot encode "沒" (0x6c92) (documented in pdf-lib #1152). Other PDF writers skip encoding checks and emit the file anyway, leaving Adobe Acrobat, Chrome PDFium, and macOS Preview rendering empty boxes, question marks, or corrupted Latin characters.

When you inspect the saved file afterward, field.getText() still returns the exact Unicode string. The text is stored inside the document, yet the visible box displays garbage or refuses to serialize. PDF separates a form field's logical data value from its drawn appearance stream, and those two structures obey different encoding rules.

Two encoding systems in one form field

Under ISO 32000-1, every AcroForm text field, dropdown, and option list maintains its state in two separate places.

The logical value sits in the field dictionary's /V entry as a PDF text string. ISO 32000-1 §7.9.2.2 allows text strings to use PDFDocEncoding (a single-byte encoding for code points 0 to 255) or UTF-16BE prefixed with the byte-order mark 0xFE 0xFF. In hex notation inside a PDF object, "田中" is stored as <FEFF75304E2D>. Any compliant PDF parser can store and extract arbitrary Unicode strings in /V.

The visual text on the page lives in the widget's normal appearance stream (/AP -> /N), which is a Form XObject content stream. Under ISO 32000-1 §12.7.3.3 (Variable Text), viewers paint the field box by running content-stream operators inside /Tx BMC ... EMC using the Default Appearance (/DA) string and the AcroForm Default Resources (/DR) font dictionary:

/Tx BMC
  q
    BT
      /Helv 12 Tf
      0 g
      2 6 Td
      (M\374ller) Tj
    ET
  Q
EMC

Content streams do not recognize UTF-16BE or PDFDocEncoding. The string operands passed to Tj or TJ are raw character codes decoded strictly through the /Encoding of whichever font /Helv points to in /DR.

Why default /Helv fields break outside WinAnsi

Most form templates generated by desktop editors initialize /DA to /Helv 12 Tf 0 g and bind /Helv in /AcroForm -> /DR -> /Font to the Standard 14 Helvetica font with /Encoding /WinAnsiEncoding.

WinAnsiEncoding maps single bytes to Windows-1252, covering 218 Latin characters such as ASCII plus Western European accented letters (ü, é, ñ). It lacks code points for Polish ł, Romanian ț, Greek, Cyrillic, Arabic, and CJK ideographs.

Three failure modes show up when non-WinAnsi text enters a /Helv field:

1. Strict libraries throw an encoding error when rebuilding /AP -> /N because the code point has no WinAnsi byte. That is why pdf-lib fails on save() if the default Helvetica appearance provider runs.
2. Permissive writers substitute ? (0x3F) into /AP or dump raw UTF-8 bytes into a single-byte stream. The viewer reads each UTF-8 byte as a separate WinAnsi character, turning multi-byte Unicode text into broken Latin pairs.
3. Within bytes 0x80 to 0x9F, WinAnsiEncoding and PDFDocEncoding disagree. Byte 0x97 represents an em dash in WinAnsi and Š in PDFDocEncoding, so mixing the two corrupts punctuation across viewers.

Setting /NeedAppearances true on /AcroForm sometimes hides the bug in Adobe Acrobat, which regenerates missing appearances with system fonts on open. Chrome PDFium, Safari, and mobile viewers either rely on /AP or ignore /NeedAppearances (deprecated in PDF 2.0), as covered in why filled PDF forms open blank in some viewers.

How Type0 composite fonts and Identity-H work

To render non-Latin or mixed-script text reliably inside /AP -> /N across viewers, the font referenced by /DA and the appearance stream's /Resources must be a Type0 composite font (/Subtype /Type0).

A Type0 font dictionary combines /Encoding /Identity-H (horizontal writing) with a /DescendantFonts CIDFont pointing to an embedded TrueType or OpenType font program and a /ToUnicode CMap stream:

25 0 obj
<<
  /Type /Font
  /Subtype /Type0
  /BaseFont /NotoSansCJK-Regular
  /Encoding /Identity-H
  /DescendantFonts [26 0 R]
  /ToUnicode 27 0 R
>>
endobj

Instead of single-byte WinAnsi literals like (Hello) Tj, the /AP -> /N stream emits two-byte glyph or character IDs as hex strings (<00280045> Tj). Because the font program and /ToUnicode CMap travel inside the PDF, every viewer renders the exact glyphs and preserves text copy-paste without relying on local OS font fallback.

Embedding a Unicode font for forms in JavaScript

In pdf-lib, filling non-ASCII forms requires registering @pdf-lib/fontkit (or @cantoo/fontkit), embedding a font that covers the target script, and passing that PDFFont instance to form.updateFieldAppearances(unicodeFont) after setting field values.

import { PDFDocument } from 'pdf-lib';
import fontkit from '@pdf-lib/fontkit';

async function fillUnicodeForm(pdfBytes, fontBytes) {
  const pdfDoc = await PDFDocument.load(pdfBytes);
  pdfDoc.registerFontkit(fontkit);

  // Embed a TTF/OTF font containing the target Unicode glyphs
  const unicodeFont = await pdfDoc.embedFont(fontBytes);

  const form = pdfDoc.getForm();
  form.getTextField('ApplicantName').setText('Łukasz Müller');
  form.getTextField('LocalAddress').setText('東京都渋谷区 1-2-3');

  // Rebuild /AP streams using the embedded Type0 font
  form.updateFieldAppearances(unicodeFont);

  // Updated fields are now clean; skip the default Helvetica pass
  return pdfDoc.save({ updateFieldAppearances: false });
}

The call order determines whether the pipeline succeeds. Calling form.updateFieldAppearances(unicodeFont) before setText() or select() fails, because mutating a field marks it dirty again and pdfDoc.save() runs form.updateFieldAppearances() with no arguments by default, falling back to StandardFonts.Helvetica.

File size and editability tradeoffs

Embedding a full CJK or pan-Unicode font adds several megabytes to each PDF. Passing { subset: true } to pdfDoc.embedFont(fontBytes, { subset: true }) strips unused glyphs when flattening forms or shipping read-only files.

Keep the full font embedded without subsetting if recipients still need to edit fields interactively. A subsetted Type0 font contains only the glyphs present when saved, so typing new characters after download produces missing boxes or inconsistent font fallback.