108 Tesseract runs on a skewed, noisy scan: the preprocessing order that held 98.5–99.5%

The usual recipe for scanned PDFs is deskew, denoise, binarize. What I couldn't find was which step actually matters, or whether the order does. I came to this from content work, not CS, so I took one scanned page and built a grid. The page is page 8 of H. G. Wells' The Time Machine (Heinemann, 1895), a public domain scan on Wikimedia Commons, 1,002 characters of ground truth. I rotated it by 0°, 1°, 3° and 5°, gave it three noise levels (none, Gaussian σ=25, salt-and-pepper 5%), and ran nine treatments on each, counting "do nothing" as one of them. That is 4 × 3 × 9 = 108 runs of Tesseract 5.5.3, one run each, PNG input, on a Mac with an Apple M4. Accuracy is 1 − edit distance / 1,002, so 99.0% means about 10 characters wrong, missing or extra. It is one clean single-column page, and the skew and noise came from a script, not a real scanner.

What to look at: the "Denoise → deskew → Otsu" row stays dark green across all twelve columns, while "Deskew only" turns red under salt-and-pepper noise and "Adaptive only" is 0.0 under Gaussian noise. Sample: page 8 of H. G. Wells, The Time Machine (1895), public domain scan via Wikimedia Commons; skew and noise are my own.

Noise hurt Tesseract far more than skew

On the clean page with no preprocessing, accuracy went 98.9 at 0°, 99.5 at 1°, 98.1 at 3° and 96.2 at 5°. So 1° cost nothing, 3° cost 0.8 points and 5° cost 2.7 points, and deskewing alone pulled the 5° page back to 99.0. With Gaussian noise it went from 97.9 at 0° to 92.9 at 5°. Salt-and-pepper was the real damage: 75.5 with the page perfectly straight, 63.3 at 5°.

Deskewing a speckled page first made it worse

With salt-and-pepper noise, "deskew only" scored 48.9, 42.2, 28.3 and 7.3 across the four angles, worse than doing nothing in every column. Deskew then denoise recovered to 91.5–94.4. Swapping the order, denoise then deskew, gave 98.6–99.1. My guess, which matches the intermediate images: the rotation uses cubic interpolation, which smears each isolated black or white pixel into a gray blob a few pixels wide. A 3×3 median can't remove blobs that size, and Tesseract reads them as strokes.

Otsu on its own stayed within 1.2 points of doing nothing on the clean and Gaussian pages, probably because Tesseract binarizes internally (an inference). Adaptive thresholding (block 31, C=15) on the Gaussian page returned empty text at all four angles.

The order that held across all twelve conditions

Denoise, then deskew, then Otsu scored between 98.5 (salt-and-pepper, 0°) and 99.5 (Gaussian, 5°) in all twelve columns. Denoise then deskew without Otsu was close behind at 98.2–99.1, so the order does most of the work. Below is a simplified sketch of that pipeline, rewritten from the grid's parameters for illustration. It is not the script that produced the numbers above (that one also enlarged the canvas so corners weren't cut), and I haven't re-run the grid with it. The denoiser follows the noise type: non-local means (h=15, template 7, search 21) for Gaussian noise, a 3×3 median otherwise. The angle comes from a projection profile.

import numpy as np
import cv2

def skew_angle(page):
    small = cv2.resize(page, None, fx=0.25, fy=0.25)
    _, ink = cv2.threshold(small, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)
    h, w = ink.shape
    def spread(a):
        m = cv2.getRotationMatrix2D((w / 2, h / 2), a, 1.0)
        return cv2.warpAffine(ink, m, (w, h)).sum(axis=1).var()
    return max(np.arange(-7, 7.01, 0.1), key=spread)

def clean_page(page, gaussian_noise=False):
    if gaussian_noise:   # 1. denoise first
        page = cv2.fastNlMeansDenoising(page, None, 15, 7, 21)
    else:
        page = cv2.medianBlur(page, 3)
    h, w = page.shape   # 2. deskew, white fill
    m = cv2.getRotationMatrix2D((w / 2, h / 2), skew_angle(page), 1.0)
    page = cv2.warpAffine(page, m, (w, h), flags=cv2.INTER_CUBIC, borderValue=255)
    return cv2.threshold(page, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]

One red cell is misleading. "Denoise only" on the clean 5° page scored 34.0. The characters weren't misread. Tesseract's layout analysis chopped the single text column into several blocks and interleaved the sentences, and the same image deskewed first scored 98.9. I haven't worked out why the layout split there, so don't read it as "median filtering hurts". The filter does hurt once the kernel outgrows the type. A 9×9 median took the straight page from 98.9 to 91.0, and on a copy scaled to half size a 5×5 median already dropped it to 85.8. The 3×3 that worked here is tied to this page's resolution.

Other recognizers wanted different steps

I also checked two other engines on a Chinese page (557 characters, from the 1986 edition of the official Simplified Character list). macOS's built-in text recognition stayed at 4–12 wrong characters through 0–5° skew and Gaussian noise, deskewing did almost nothing, and Otsu made it slightly worse on the clean and Gaussian pages (salt-and-pepper was the one noise that clearly hurt it). The third engine was the PDF content extraction in ImgIng https://imging.ai/ , which runs a PP-OCR model inside the desktop browser. It was the most skew-sensitive: fine at 1°, lines out of order from 3°, and it lost two lines at 3° and roughly half the page at 5°, while the interface still showed the job as finished with no warning. After denoise and deskew the same 3° noisy Chinese page came back to 2 wrong characters. So the grid answers the question for Tesseract only.

If I set this up again: denoise, deskew, then Otsu, with a kernel sized to the text, and before adding a step I run one page through my actual recognizer with and without it. I haven't tested real skewed scans, two-column layouts, phone photos or low-DPI pages, so this is about order of operations, not real-world accuracy.

Sources: H. G. Wells, The Time Machine, Heinemann 1895, p. 8, public domain scan via Wikimedia Commons, ground truth from English Wikisource (CC BY-SA 4.0); Chinese page from the 1986 Simplified Character list, Internet Archive, Public Domain Mark 1.0. Rotated and noisy versions are my own.