Table OCR to Excel: wrapped cells had no cheap fix, 11 of 34 still wrong on an unruled sample
Picture a freelance request: the client has a work-order table, but what arrives is a screenshot, and the job is to get it into a spreadsheet. OCR reads the characters fine and the columns break. On a self-made 7 × 5 sam
Picture a freelance request: the client has a work-order table, but what arrives is a screenshot, and the job is to get it into a spreadsheet. OCR reads the characters fine and the columns break. On a self-made 7 × 5 sample with six two-line cells, rebuilding the grid from box coordinates left 4 of 34 cells wrong when the table had ruling lines and 11 of 34 when it did not. Cutting along the ruling lines got all 34 right, which needs the lines to be there.
What was tested
The sample is a fictional work-order table drawn for this test, in Chinese: 7 rows, 5 columns, 34 non-empty cells, six of them holding two lines of text. There are two renderings of the same data, one with ruling lines and one without, both clean and unskewed, so everything below is about these two images. Recognition ran through ImgIng https://imging.ai/ (default tier, desktop browser, Apple-silicon Mac), and the exports were the TXT plus a JSON with four corner coordinates per text box. Each image was recognised once; the ruled one ran twice with identical output.
Nothing was pasted into a real spreadsheet. A script split the exported text on newlines and Tabs and compared each cell with the ground truth, and a cell counts as wrong when it is not in its original row and column. The plain export put every row into 2 cells instead of 5, so 34 of 34 were misplaced on both versions, while the text and the row count of 7 were right.
Every text line becomes its own box
The JSON has 40 boxes for 34 cells. The extra six are the second lines of the wrapped cells, and nothing in a box says it is the lower half of the cell above. The rebuild groups boxes into rows by centre y with a tolerance of 0.5 × box height, projects them onto the x axis to find column gaps (empty in at least 80% of rows, at least 0.2 × box height wide), then applies one rule for wrapping: a row whose first column is empty is folded into the row above. The gap width was tuned after seeing results, so it is a fitted number. Below is the wrap rule as a simplified sketch written from its definition; the figures in this post come from the full script.
stitched = []
for band in bands: # boxes grouped by centre y, top to bottom
has_key = any(lane_of(b) == 0 for b in band)
if stitched and not has_key:
stitched[-1].extend(band) # continuation of the row above
else:
stitched.append(band)
On the ruled image this left 4 of 34 wrong, and all four were wrapped cells whose two lines came out in reversed order. On the unruled image it left 11 of 34 wrong, because the rule joined the second line of one row to the first line of the next row. The cause was not traced, and a top-down fold evidently does not hold on this sample.
Ruling lines recovered all 34
The other approach ignores text positions. Binarise the image, extract the horizontal and vertical lines, treat each white region they enclose as a cell, and drop every text box into the cell holding its centre. On the ruled table this detected 35 cells (the 7 × 5 grid), restored all six wrapped cells as two lines each, and left 0 of 34 wrong. The unruled version has nothing to cut along, so its 11 stays 11.
When to stop and ask for a ruled export
For a one-person shop the accounting is short. 11 of 34 is roughly one cell in three, so somebody still has to walk every row against the image. If the image has ruling lines, cut along them. If it has none, one message to the sender asking for an export with gridlines, or for the original file, probably costs less than more threshold tuning. Unruled plus wrapped is a combination for which this test found no reliable method.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.