Your usability test is incomplete if you can't see how the tester reacts
This is a submission for the Presage SmartSpectra SDK Challenge What I Built When you run a usability test, you see what people click. You miss the moment they get annoyed, confused or worried, and testers of
This is a submission for the Presage SmartSpectra SDK Challenge
What I Built
When you run a usability test, you see what people click. You miss the moment they get annoyed, confused or worried, and testers often don't say it.
I built Flinch to catch that moment. It is a desktop app. The tester does a task on a website inside the app, and a normal webcam reads their pulse, breathing, blinking and facial expression with the Presage SmartSpectra SDK. When these signals change, Flinch marks the moment. After the test you get a report with one card for each moment: the screen at that time, the signals that changed, the clicks just before, and an optional AI note.
Flinch is for people who run usability tests: designers, developers and small teams. You need only a normal webcam. I used the one in my laptop.
One test, step by step
The pictures below come from one real test that I did on myself, in the demo shop that comes with Flinch. The task was: "Buy one mug and pick it up at the studio."
- Set up. Pick the demo shop or type the address of your own site, and write the task.
- Consent. The tester reads one page in plain words about what is measured and what is kept. The camera stays off until they agree.
- Camera check. Flinch waits until SmartSpectra reports a stable pulse. In my test this took about 17 seconds.
- 30 seconds of calm. This gives the tester's own resting level for today.
- The task. The site runs inside the app, and a panel on the right shows the live signals.
- The report. A timeline, and one card for each reaction moment.
This is step 5, 1 minute and 8 seconds into my test. The shop has just told me that my card may have been charged twice. The panel shows a pulse of 85 bpm, and the face reading is "surprise", 99% sure. In the app, the top of the panel shows the tester's camera picture. I replaced mine with a grey figure for this post.
This is the top of the report for the same test. The test was 1 minute and 25 seconds long. My pulse was 75 bpm at rest and 82 bpm on average during the task. Each numbered pin is a reaction moment, and the green pin is a positive one.
The report card
Under the timeline there is one card for each moment. For me the cards are the heart of Flinch.
This is the card for the "Payment failed" moment. Everything about the moment is in one place:
- The screen. A screenshot of the page at that time.
- The signals. What changed, with the numbers. Here the face read as "sad" for 3.5 seconds and then as "surprise" for 2.9 seconds. The card also lists a change in the breathing rate.
- The clicks. What happened on the page just before. Here there are three clicks on "Pay now", because the button gives no sign that it is working.
- The AI note. This part is optional. Claude looks at the screenshot and the clicks, and writes what probably caused the reaction and one fix to try.
So I do not have to put a screen recording, a click log and my own notes next to each other. The card has already done that.
Flinch found five moments in this test:
| Time | On the screen | Signals |
|---|---|---|
| 0:07 | A newsletter popup with a tiny close button | Face "sad" 58%, and later pulse +7 bpm |
| 0:36 | The error "Invalid input." | Face "sad" 85%, then "angry" 61% |
| 0:55 | A $4.90 service fee on the payment page | Face "angry" 85% |
| 1:08 | "Payment failed", with a "charged twice" warning | Face "sad" 86%, then "surprise" 99%, and breathing +6 per minute |
| 1:22 | A free gift opens | Face "happy" 96%. A positive moment, shown in green |
Building the trap shop
To test a tool that looks for reactions, I needed a site that causes them. So I built Kiln & Co., a small mug shop where things go wrong on purpose. It has seven traps: a popup with a tiny close button, delivery options hidden behind a small link, an error that says only "Invalid input.", a form that clears your email when a check fails, a fee that shows up only on the payment page, a "Pay now" button that does nothing for four seconds, and then a "Payment failed" dialog that says your card may have been charged twice. That last one is not true, and the shop takes no real payment.
It is a strange job to build a bad shop on purpose, and my first version was too bad. In my first full test I got my own "Invalid input." error four times. I had written the phone rule myself, and I still could not pass my own form. So the phone field now shows an example number, and the error is as vague as before.
I also wanted one good moment. On the last page you open a gift box and you get a free mini mug with a face, and confetti. Flinch shows this moment in green.
Demo
The video is two and a half minutes long, with captions and no sound. The live SmartSpectra signals are on screen from 0:21, and the report starts at 1:35. The red line under the face value is not part of the app. I added it in the edit, so it is easier to see when the face reacts.
Code
asi-mir
/
flinch
See the exact screen where your tester's body reacted. Webcam-based usability testing with the Presage SmartSpectra SDK.
Flinch
Watch someone use your site. See the exact screen where their body reacted.
Flinch is a desktop app for usability testing. A tester does a task on a website inside the app. A normal webcam reads their pulse, breathing, blinking and facial expression with the Presage SmartSpectra SDK. When those signals change against the tester's own resting level, Flinch marks the moment and shows you the screen, the clicks before it, and the numbers behind it.
No video of the tester is ever recorded. Only numbers are kept, on your computer.
Run it in 3 steps
You need Node.js 20 or newer and macOS on Apple Silicon, Windows x64 or Linux (the platforms the SmartSpectra Node SDK supports).
git clone <this repo> && cd flinch npm install cp .env.example .env # then put your SmartSpectra API key in .env npm start
Get a free trial keyβ¦
The README has the setup steps. You need Node.js 20 or newer and a free SmartSpectra trial key. I built and tested Flinch on a Mac with Apple Silicon only. Without a camera or a key, npm run preview runs the whole app with made-up signals.
Flinch is built with the SmartSpectra Node SDK and Electron, and the optional AI notes use the Anthropic SDK.
How It Responds to Human Signals
Flinch uses these live signals from SmartSpectra: pulse rate, breathing rate, the chest breathing trace, blinking and facial expression.
During the test, the signals do three things. The test cannot start until SmartSpectra reports a stable pulse. The panel shows the pulse, the breathing and the face reading live. And when SmartSpectra reports that the picture is not usable, for example because the chest is out of view, a "Measurement paused" notice appears over the page and tells the tester what to fix.
After the test, the signals decide what is in the report.
How a reaction moment is found
Flinch compares the tester with themselves. The 30 calm seconds give the resting level, and then a few simple rules look for changes. Two examples:
- Face. The scores for angry, fear, disgust and contempt together reach 35% or more, at least 25 points above the resting level, for 0.7 seconds. Sad, surprise and happy have their own checks.
- Pulse. The pulse rate is 6 bpm above the median of its own readings from 30 to 10 seconds before.
There are similar rules for the breathing rate, a held breath and blinking. Changes that are close in time join into one moment. A card says "Two or more signals agree" when different kinds of signal changed, and "One signal only" when one kind changed.
The face is read about 30 times a second, so it gives the exact time and the right screenshot. All of this is plain arithmetic in one file, and no AI decides where a moment is.
Real recordings changed my first rules. When I looked down at the keyboard, the SDK read my face as "sad". So "sad" is now ignored around typing.
In my test, all five moments started with the face. The pulse joined one moment and the breathing rate joined one. The blink rule and the held-breath rule found nothing.
The SDK integration
Flinch uses @smartspectra/node-sdk in Electron. In the main process, one call connects the SDK's native runtime to the app window: bindSmartSpectraIpc(mainWindow). The window asks for the metrics and listens (shortened from src/renderer/signals.js):
const sdk = new SmartSpectraSDK({
apiKey,
requestedMetrics: [
...breathingMetrics,
...cardioMetrics,
MetricType.BLINKING,
MetricType.TALKING,
MetricType.EXPRESSIONS,
],
enableTelemetry: false,
});
sdk.on('metrics', (buf) => handlers.onSamples(normalizeMetrics(decodeMetrics(buf))));
sdk.on('validationStatus', (code, _ts, hint) =>
handlers.onValidation(code === ValidationCode.kOk, hint || ''));
await sdk.start();
Each metrics packet becomes plain samples with a time. validationStatus tells Flinch if the picture is usable. Flinch uses only the pulse and breathing samples that SmartSpectra itself marks as stable.
What it cannot do
- A moment is a hint, not proof. It shows where the body changed, not why. You still look at the screen and ask the tester.
- I tested it on one person: me. And I built the traps, so I knew what was coming.
- The camera must see the face, the shoulders and the upper chest. In my first full test I leaned in to type, my chest left the picture, and only 3% of the task was measured.
- Pulse and breathing rate are slow. They are averages over 12 and 30 seconds, so they can point only to a window of time. On the card above, the breathing "rise" from 17 to 22 is my breathing rate going back to its resting level of 22.
- The AI note can be wrong. On the "Invalid input." card, the note blamed an empty Email field. But I had typed my email. The shop had cleared it when the check failed, and the real cause was the format of the phone number.
Prize Categories
Best Thoughtful Real-World Use. The need: in a usability test you see the clicks, but not the moment when a person reacts. Flinch is for designers, developers and small teams who run such tests, and it needs only a normal webcam. To make it usable for them, one card puts the screen, the signals and the clicks together, and the tester gets a plain consent page before the camera starts.
Most Creative or Delightful Use. I wanted to see the moment that a tester does not say out loud. The unexpected part is how the body signals are used: they are not the product, they are a tool that points at the screens where the tester reacted, in a good way or a bad way. The playful part is Kiln & Co., a mug shop that I built to go wrong on purpose, with seven traps and one free gift at the end.
Consent and Data Handling
- Consent. Before the camera starts, the tester reads one page in plain words about what is measured, what is kept and what can leave the computer. The camera starts only after they tick "I have read this and I agree to take part." During the test, a "Stop and delete" button stays on screen. It ends the test and keeps nothing.
- No video. SmartSpectra processes the camera frames on the computer (Presage docs). Flinch never saves or sends a camera image.
- Storage. The signal numbers, the clicks and screenshots of the tested page are saved on the computer of the person who runs the test. A screenshot shows the page as the tester saw it, so text typed into a form can be in the picture.
- Sharing. None. Flinch has no server and no account. Only the next two points send anything out.
-
SDK telemetry. Off (
enableTelemetry: false). The SDK still signs in to Presage with the API key. - AI notes (optional). Off unless you add an Anthropic API key. Then, only when you press "Write AI notes" and say yes, Flinch sends this to Anthropic: screenshots of the tested page, the page events (such as clicks and errors), the signal changes as short text, the task and the site address. Camera images are never sent.
- Retention and deletion. A test stays until someone deletes it. "Delete" in a report removes that test, and "Delete all tests" removes everything.
Flinch is not a medical tool, and it makes no claim about anyone's health. It is made only for tests that a person has agreed to take.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.

