Measuring face symmetry in the browser with MediaPipe's 478 landmarks (no upload)

Most "face symmetry" tools upload your photo to a server and send back a number. I wanted to see how far I could get with everything running on the device: the photo never leaves the browser, and the measurement is plain geometry you can read and check.

This post walks through the approach: getting landmarks with MediaPipe, fixing head tilt, refusing to score a turned head, and turning raw distances into a score that actually separates one face from another.

1. Getting 478 landmarks in the browser

MediaPipe's Face Landmarker (@mediapipe/tasks-vision) runs as WebAssembly and returns 478 points per face: 468 for the face mesh and 10 for the irises.

import { FaceLandmarker, FilesetResolver } from '@mediapipe/tasks-vision';

const vision = await FilesetResolver.forVisionTasks(
  'https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/wasm'
);

const landmarker = await FaceLandmarker.createFromOptions(vision, {
  baseOptions: {
    modelAssetPath:
      'https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task',
    delegate: 'GPU', // fall back to 'CPU' in a catch block
  },
  runningMode: 'IMAGE',
  numFaces: 1,
});

const result = landmarker.detect(canvas); // canvas holds the user's photo
const landmarks = result.faceLandmarks[0]; // 478 points, or undefined

Two practical notes:

• Pin the version. The WASM files on the CDN must match the package version, or you get confusing runtime errors.
• The GPU delegate fails on some devices. Wrap it in a try/catch and retry with delegate: 'CPU'.

2. Convert to pixel space first

The landmarks come back normalised: x is divided by the image width and y by the image height. On anything that isn't square, one unit of x is not the same length as one unit of y, so every distance and angle is skewed. Convert before measuring anything:

const toPixels = (points, width, height) =>
  points.map((p) => ({ x: p.x * width, y: p.y * height, z: p.z }));

It's a one-liner, and skipping it quietly breaks every ratio you compute later.

3. Head tilt: correct it, don't reject it

People rarely hold the camera perfectly level. A tilted head is just a 2D rotation, so it can be undone. I measure the roll from the two iris centres (landmarks 468 and 473), then rotate every point around the middle of the face:

const leftIris = points[468].x < points[473].x ? points[468] : points[473];
const rightIris = leftIris === points[468] ? points[473] : points[468];
const roll = Math.atan2(rightIris.y - leftIris.y, rightIris.x - leftIris.x);

const cx = (points[234].x + points[454].x) / 2;
const cy = (points[234].y + points[454].y) / 2;
const cos = Math.cos(-roll);
const sin = Math.sin(-roll);

const levelled = points.map((p) => ({
  x: cx + (p.x - cx) * cos - (p.y - cy) * sin,
  y: cy + (p.x - cx) * sin + (p.y - cy) * cos,
  z: p.z,
}));

I use the irises rather than the inner eye corners because iris centres are detected more consistently, and eye shape doesn't pull them around.

4. Head turn: refuse to score it

A turned head (yaw) is different. It isn't a flat rotation; one half of the face is foreshortened, and that looks exactly like asymmetry. You can't undo it from a single 2D photo, so the honest option is not to score it.

To detect it, I compare the horizontal distance from the nose bridge to each side of the face:

const bridge = points[168];
const toLeft = Math.abs(bridge.x - points[234].x);
const toRight = Math.abs(bridge.x - points[454].x);
const yawRatio = (toLeft - toRight) / (toLeft + toRight);

const usable = Math.abs(yawRatio) <= 0.15;

My first version used the nose tip (landmark 1). Bad idea: the tip sticks out of the face, so at larger angles its position stops moving in one direction and starts to reverse. The bridge (168) sits on the face plane and moves steadily with the turn. If the ratio is over 0.15, the tool asks for a straight-on photo instead of showing a misleading number.

5. The symmetry measurement

With a level, front-facing set of points, the measurement itself is simple:

1. Draw a midline from the nose bridge (168) to the chin (152).
2. Take mirrored pairs: outer eye corners (33, 263), mouth corners (61, 291), cheeks (234, 454) and brows (70, 300).
3. For each pair, compare how far each point sits from the midline.
4. Average the differences and divide by face width, so the result doesn't depend on photo size.

const top = points[168];
const bottom = points[152];
const faceWidth = dist(points[234], points[454]);

// Perpendicular distance from point p to the line top-bottom
const distToMidline = (p) =>
  Math.abs((top.x - p.x) * (bottom.y - p.y) - (top.y - p.y) * (bottom.x - p.x)) /
  dist(top, bottom);

const pairs = [[33, 263], [61, 291], [234, 454], [70, 300]];

const avgDiff =
  pairs.reduce(
    (sum, [l, r]) => sum + Math.abs(distToMidline(points[l]) - distToMidline(points[r])),
    0
  ) / pairs.length;

const deviationPercent = (avgDiff / faceWidth) * 100;

6. Turning a deviation into a score

This is where most tools quietly make everyone look good. Real faces are all fairly symmetric: the deviation usually lands somewhere between about 0.5% and 5% of face width. A linear "100 minus deviation" puts nearly everyone between 95 and 99, which tells you nothing.

I use an exponential curve instead:

const score = Math.max(0, Math.min(100, 100 * Math.exp(-deviationPercent / 2.8)));

The constant 2.8 was chosen by running real front-facing photos through the pipeline and checking that the spread was useful. A score above 80 needs a deviation under about 0.6%, which is rare. The middle of the range (roughly 30 to 55) is where typical faces end up. Small differences near the top still matter, and large deviations stay distinguishable instead of all collapsing to zero.

The trade-off: scores look "low" compared with tools that report 90%+ for everyone. A 40 here and a 90% elsewhere can describe the same face. I'd rather the number mean something.

What I learned

• Camera and pose matter more than people expect. A slight turn, a wide-angle selfie lens held close, or uneven lighting can move the result more than real facial differences. I now ask for a straight-on photo at arm's length or further.
• Measure the same face twice. Testing two photos of the same person was the quickest way to find unstable landmarks and bad formulas.
• Be clear about what a 2D number is. It's a measurement of one photo, not a verdict on a face.

If you want to try the result, I put this together as a free face symmetry test that runs entirely in your browser; the photo is never uploaded. I'm happy to answer questions about the landmark choices or the scoring curve in the comments.