Math.round(x * 100) / 100 rounds 1.005 to 1: the rounding bug in our unit tables

We build free calculators for people who pull-test anchors and fasteners. One of them converts displacement between mm and inches and prints two reference tables. On 2026-11-04 our publish gate failed the page: the 1/16 in row said 1.587 mm.

1/16 in is 0.0625 in, and 1 in is exactly 25.4 mm, so the exact value is 1.5875 mm. Rounded half up to three decimals that is 1.588. The table was off by one in the last digit, on a page whose whole job is to be right about the last digit.

Where 1.587 came from

The static tables were generated in Python with an f-string:

>>> f"{0.0625 * 25.4:.3f}"
'1.587'
>>> from decimal import Decimal
>>> Decimal(1.5875)
Decimal('1.587499999999999911182158029987476766109466552734375')

1.5875 has no exact binary form. The nearest double is a little below it, and formatting rounds that stored value correctly: it is below the halfway point, so it goes down. Python's own docs give the same example with round(2.675, 2), which returns 2.67. [src: https://docs.python.org/3/library/functions.html#round]

The fix for the tables was to stop using floats for them: build each value as a Decimal from strings and round with ROUND_HALF_UP.

from decimal import Decimal as D, ROUND_HALF_UP
f"{(D('0.0625') * D('25.4')).quantize(D('0.001'), ROUND_HALF_UP)}"   # '1.588'

The same class in the browser

The live calculators round in JavaScript, so we checked those next. They used the two common idioms, and both have the same problem:

Math.round(1.005 * 100) / 100   // 1, not 1.01
1.005 * 100                     // 100.49999999999999
(1.005).toFixed(2)              // "1.00"
(2.55).toFixed(1)               // "2.5"

1.005 * 100 lands just below 100.5, so Math.round goes down. toFixed rounds the stored binary value, which for 1.005 and 2.55 is just below the halfway point; MDN documents the 2.55 case. [src: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toFixed]

There is a second, smaller trap: Math.round rounds halves toward positive infinity, so Math.round(-2.5) is -2, not -3, and Math.round(-0.4) returns -0. [src: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math/round]

All 19 calculator pages we had at the time rounded with Math.round(x * 10^d) / 10^d or toFixed.

The fix: round the decimal string, not the binary value

JavaScript prints a number using the shortest decimal string that maps back to the same double. For the double nearest 1.005 that string is "1.005". If you shift the decimal point by editing the exponent in that string, instead of multiplying, the number you round is the one you see:

function rd(x, d) {
  if (!isFinite(x)) return x;
  var s = x < 0 ? -1 : 1, p = String(Math.abs(x)).split("e");
  var r = Math.round(Number(p[0] + "e" + ((+p[1] || 0) + d)));
  if (r === 0) return 0;
  p = String(r).split("e");
  return s * Number(p[0] + "e" + ((+p[1] || 0) - d));
}

rd(1.005, 2)    // 1.01
rd(-1.005, 2)   // -1.01   (half away from zero, like ROUND_HALF_UP)
rd(-2.5, 0)     // -3
rd(-0.004, 2)   // 0, not -0

Number("1.005e2") is exactly 100.5, so Math.round sees the true halfway value. The sign is handled separately, which gives half away from zero for negatives and matches Python's ROUND_HALF_UP on the table side. Exponents in the string (1e-7) are handled by adding to the existing exponent. The result goes to toLocaleString for display; the calculation itself keeps full precision.

Making it stay fixed

A fix that depends on everyone remembering it does not last, so the repo now has two tests:

• A lint test that reads every calculator page and fails on Math.round(x * 10...) / Math.round(x * Math.pow(10, d)) or a bare .toFixed( (one on a value rd already rounded is allowed). It flagged all 19 old pages before the change and none after.
• A headless-browser check that loads each live page, pulls out the rd that page actually ships, and runs 14 fixed cases through it (1.005, 1.5875, 0.0625 * 25.4, 2.675, -1.005, 1e-7, 0.1 + 0.2 and others), plus one check in the displacement converter's UI. It ran 267 checks when we made the change and 295 now, after two more pages, all passing.

We also rechecked the 245 static table cells on six pages against exact Decimal values after the 1/16 in fix: none were wrong.

What it does not fix

rd rounds the number it is given. If the arithmetic before it already drifted, rd rounds the drifted value: in our stress converter, 0.00100005 ksi to psi computes 1.0000499999999999, not 1.00005, so rd shows 1. We left that case out of the test on purpose and wrote down why. It fixes halfway rounding, not every floating-point error. For values that must be exact (prices, published tables), compute in decimal from the start.

The part we did not expect

The bug was not found by a unit test. It was found because the page went through our publish gate, which checks each claim and number in a draft against a facts file before it goes live, and the gate's claim-by-claim check flagged the row. A generalized version of that gate is the Publish Gate we sell ($19 one-time). A free claim-check skill for Claude Code, /verify-claims, is in our MIT-licensed agent-ops-kit-lite repo. The calculators, including the fixed displacement unit converter, are free at calyx-ai.net/qa.

Written by Calyx AI, a product line of Quality Tester Solutions LLC. Drafted with AI assistance and checked by the gate it describes; the code and the numbers are from our own repo and were run on Node v26.7.0 and Python 3.14.8.