Each OCR engine had its own minimum text height, and none had a maximum up to 60 px
When a screenshot is too small for OCR, upscaling it first is the tempting move. After three years on video codecs that bothers me: the detail was spent, or not, at capture time. So here is a test of how much text height
When a screenshot is too small for OCR, upscaling it first is the tempting move. After three years on video codecs that bothers me: the detail was spent, or not, at capture time. So here is a test of how much text height each recognizer needs, and what upscaling recovers below it.
The samples are deliberately clean: a self-written Chinese passage of 275 characters and an English one of 452, black on white, single column, rendered directly at each height. Text height is the ink bounding box of ε½ for Chinese and of a capital H for English. Nine heights from 6 to 60 px, two Chinese fonts: Songti SC, a serif with thin horizontals, and Heiti SC, a sans. Readers were macOS built-in text recognition in accurate mode, Tesseract 5.5.3 on English only, and the image text recognition in ImgIng https://imging.ai/ , run in a desktop browser. One run each on an Apple M4 Mac. Accuracy is 1 minus character edit distance over length; one wrong Chinese character costs 0.36 points.
The floor is different for every recognizer
| Text height | macOS Songti | macOS Heiti | ImgIng Songti | ImgIng Heiti | Tesseract English |
|---|---|---|---|---|---|
| 6 px | 1.82 | 6.91 | 71.27 | 95.27 | 16.81 |
| 8 px | 92.0 | 88.0 | 97.82 | 98.55 | 95.35 |
| 10 px | 96.0 | 99.64 | 98.55 | 100 | 99.34 |
| 12 px | 99.64 | 100 | 100 | 100 | 99.34 |
| 16 px | 100 | 100 | 100 | 100 | 100 |
| 20 px | 100 | 100 | 100 | 100 | 100 |
| 30 px | 99.64 | 100 | 100 | 100 | 100 |
| 40 px | 99.64 | 100 | 100 | 100 | 100 |
| 60 px | 99.27 | 99.64 | 100 | 100 | 100 |
Every column hits a cliff at a different row. The browser recognizer was error-free on Chinese from 12 px in Songti and 10 px in Heiti. macOS was clean from 12 px in Heiti, but Songti only reached zero errors at 16 px. Tesseract needed 16 px for clean English. From 12 to 40 px both Chinese recognizers sat between 99.64% and 100%, and macOS Songti dipped to 99.27% at 60 px; 8 px is where they split. At 6 px macOS is essentially gone. Near the floor the font matters about as much as the engine: at 10 px macOS made 11 errors on Songti and 1 on Heiti, which fits Songti's horizontals shrinking to about a pixel.
Nothing got worse up to 60 px
Large text didn't hurt in this range. The browser recognizer stayed at 100% from 12 to 60 px on both fonts, Tesseract at zero errors from 16 to 60. The macOS losses at 30, 40 and 60 px are all one event: the last character of the first line went missing, a line-end drop rather than a misread glyph. The useful number is a floor, not a sweet spot. The test stopped at 60 px, so where a real ceiling starts is still open.
Upscaling won back part of the gap
Then the obvious fix: resizing the 6 to 12 px images to roughly 32 px text height with bilinear, bicubic and Lanczos. It suits Tesseract: English at 8 px went from 21 errors to 0 with bilinear. macOS recovered a lot at 6 px, Heiti from 256 errors to 37, yet Songti at 6 px peaked at 53.1% while native 12 px scored 99.64%. The browser recognizer barely moved, and Lanczos often made it worse, Songti at 6 px going from 79 errors to 133. Bilinear was best or tied most often; I suspect Lanczos ringing on stroke edges but haven't checked. Interpolation moves a small image into the scale an engine expects. My reading, not something this test isolated, is that it can't restore strokes that already merged at six rows tall, any more than upscaling a 360p encode recovers what the encoder dropped.
One side note: the same 8 px Heiti image pasted unscaled onto a 1200Γ1200 white canvas dropped macOS from 88.0% to 0%, so for that engine the size of the whole image seems to matter too; why is only a guess.
Measure the text height first
This all needs the text height in hand, and two cheap estimators held up. Horizontal projection binarizes with Otsu and takes the median height of ink-bearing row runs. Connected components takes the 90th percentile of component heights, since Chinese characters split into parts. On the Chinese renders both landed within Β±2 px up to 30 px. On English, projection measures the whole line, about 1.37 times cap height, so trust connected components there. Ruled tables break projection: one bordered table came out as a single 559 px line, connected components gave 23. A self-made slide screenshot at device pixel ratio 1 measured about 19 to 20 px, a chat screen at ratio 3 about 41 to 45 px.
A sensible order from these numbers: measure first, and above about 16 px leave the image alone. If you control capture, take it larger with a higher device pixel ratio or zoom, since native 12 px beat every upscaled 6 px result here. If not, and Chinese text is under 12 px, consider upscaling to about 30 px with bilinear and compare one output with the original rather than assume it helped. These were clean renders, so real screenshots and photos may do worse, and single runs can shift by a character or two.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.