Your End-to-End Tests Are Green. So Why Are Users Still Losing Data?
Imagine this scenario. A user opens a web application, edits a document, clicks Save, and sees a reassuring green checkmark. Everything looks fine. They close the tab, come back ten minutes later, and discover that th
Imagine this scenario.
A user opens a web application, edits a document, clicks Save, and sees a reassuring green checkmark.
Everything looks fine.
They close the tab, come back ten minutes later, and discover that their changes are gone.
The QA team investigates. The automated test for saving documents passed. The API returned HTTP 200. The UI displayed "Saved." The CI pipeline was green.
So what happened?
Maybe the application displayed the success message before the server confirmed persistence. Maybe two requests arrived out of order. Or maybe the browser closed while an optimistic update was still waiting for confirmation.
These bugs are surprisingly easy to miss because most automated tests focus on whether an action succeeded, not whether the application ended up in the correct state.
And that's becoming a bigger problem as web applications increasingly rely on autosave, optimistic updates, background synchronization, and asynchronous operations.
The problem with testing only what you can see
Consider a simple document editor.
When you change the title of a document, several things might happen:
- The title changes immediately in the browser.
- The application marks the document as modified.
- A background request sends the new title to the server.
- The server validates and stores the change.
- The application receives confirmation.
- The UI displays "Saved."
These operations don't necessarily happen instantly or in a predictable order.
Now imagine that the user clicks Undo between steps 3 and 4.
Which title should the server store?
What happens if the original request completes after the undo operation?
What if the server rejects the update?
A basic browser test that fills an input and checks its value won't catch any of these problems.
The better approach is to test the transitions between states, not just the final screenshot.
There's a useful walkthrough of this problem in How to Test Undo, Redo, and Optimistic Reconciliation Flows in a Browser App Without Brittle State Assertions.
The important distinction is between what the user sees and what the application actually persists.
You don't necessarily need access to the application's internal state. In many cases, observing the UI and verifying the relevant network requests is enough.
A better way to test autosave
Let's say we're testing a note-taking application that saves changes automatically.
Instead of testing only the happy path, we should cover a few different situations.
| Scenario | What the test should verify |
|---|---|
| Normal save | The change is persisted and survives a reload |
| Slow API response | The UI distinguishes pending changes from saved changes |
| API failure | The application doesn't falsely report success |
| Undo during save | The final persisted state matches the intended action |
| Two rapid edits | An older response doesn't overwrite a newer change |
| Server normalization | The UI reflects the authoritative server response |
Playwright can help by intercepting requests and controlling server responses.
Here's a simplified example:
import { test, expect } from '@playwright/test';
test('reconciles an optimistic edit with the server', async ({ page }) => {
let releaseResponse;
const responseGate = new Promise(resolve => {
releaseResponse = resolve;
});
await page.route('**/api/notes/42', async route => {
if (route.request().method() !== 'PATCH') {
return route.continue();
}
await responseGate;
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
id: 42,
title: 'Release notes',
revision: 8
})
});
});
await page.goto('/notes/42');
const title = page.getByRole('textbox', {
name: 'Title'
});
await title.fill(' Release notes ');
// The local edit is visible before confirmation.
await expect(title).toHaveValue(' Release notes ');
// The application should indicate pending persistence.
await expect(page.getByText('Saving...')).toBeVisible();
// Allow the server response to arrive.
releaseResponse();
// The server-normalized value becomes authoritative.
await expect(title).toHaveValue('Release notes');
await expect(page.getByText('Saved', { exact: true })).toBeVisible();
});
This is an illustrative test for an application with those specific autosave behaviors. The selectors, endpoint, and response format need to match your application.
The interesting part is that we're deliberately holding the API response.
That allows us to verify the optimistic state before the server confirms the change.
We can extend the same pattern to return a 409 conflict, a 500 error, or an older revision.
One important addition: after a successful save, reload the page and verify the persisted value using the real backend. Mocked API responses prove that the frontend handles a particular response correctly, but they don't prove that the backend stored anything.
That's a separate contract worth testing.
Don't forget the browser itself
Here's another situation that's easy to overlook.
The user edits a form and closes the tab before saving.
A well-designed application might warn them that they have unsaved changes.
Many applications implement this behavior using the browser's beforeunload event.
The problem is that this isn't a normal JavaScript confirmation modal. The browser controls the dialog, and the behavior can vary depending on the browser and the circumstances.
A test that tries to read the exact dialog text or click its buttons using fixed delays can become unnecessarily flaky.
A better strategy is to verify the conditions under which the warning should exist:
- Does editing the form activate the unsaved-changes protection?
- Does a successful save remove that protection?
- Does navigation behave correctly when there are no pending changes?
- Does the application preserve or recover data when saving fails?
For the browser-specific details, How to Test beforeunload Confirmation Flows Without Flaky Dialog Handling explains how to focus on the behavior rather than fragile dialog handling.
And remember that beforeunload is not a guarantee against data loss. Browsers don't reliably fire it in every situation, particularly on mobile devices.
If protecting user data matters, the application also needs a sound persistence and recovery strategy.
The API can pass while the product is broken
There's another common mistake in test automation: treating a successful HTTP status code as proof that the operation worked.
Consider this response:
{
"status": "success",
"documentId": 42,
"revision": 7
}
The request returned 200.
But what if the client was expecting revision 8?
What if the API response is missing a required field?
What if the response is valid JSON but violates the contract between the frontend and backend?
This is where API contract testing becomes important.
Instead of asserting only the status code, we should validate the response schema, required fields, revision semantics, and error handling.
The browser test can verify what the user experiences. The API test can verify the underlying contract.
They complement each other.
For teams deciding how to implement this, API Tools That Fit Schema Checks, CI Reports, and Day-to-Day Handoff compares approaches using tools such as Postman, Bruno, Karate, and REST Assured.
The choice of tool matters less than having automated contract checks that run consistently and are understandable to the people maintaining them.
A small collection of reliable API tests can catch problems that dozens of browser-only tests might miss.
A failed test is useful only if someone can understand it
Let's assume we've implemented all these scenarios.
One morning, CI reports that the autosave test failed.
Great. We caught a regression before production.
But now someone needs to investigate.
What evidence do they have?
A message saying:
Expected "Saved" to be visible. Timed out after 5000ms.
That's not particularly helpful.
Was the API slow? Did it return an error? Did a JavaScript exception prevent the UI from updating? Was the test itself waiting for the wrong element?
These are very different problems.
A useful automated test failure should preserve enough information to distinguish them.
Ideally, the engineer investigating the failure can inspect the relevant screenshots, browser console output, network activity, execution steps, and environment information.
This is why the ability to debug failures should be part of how teams evaluate their browser testing infrastructure.
The article Evidence, Replay, and Triage: A Benchmark Plan for Browser Testing Platforms proposes evaluating platforms by evidence quality, replayability, and the time required to investigate failures.
I think this is an underrated way to evaluate automation.
Teams often compare tools by how quickly they can create tests or how many tests they can execute in parallel.
But execution speed is only one part of the equation.
If a test takes 20 seconds to execute and 45 minutes to investigate when it fails, making that test run twice as fast probably isn't your biggest opportunity for improvement.
Improving the evidence collected during execution might save considerably more engineering time.
The last problem: who decides whether the release is safe?
Now imagine that the autosave tests are passing again.
The API contract tests are green.
The browser tests are green.
Can we release?
Maybe.
But what happens if one important scenario was skipped? Or if a known bug affecting data persistence was accepted temporarily?
An automated test report can tell us what passed and failed. It doesn't necessarily tell us whether the release is acceptable.
That requires a connection between requirements, test coverage, known defects, and the release decision.
For small projects, this might be a lightweight checklist attached to a pull request.
For larger organizations, it often involves a test management system, issue tracking, and formal approvals.
The useful question isn't whether a tool can display a list of test cases.
It's whether someone can answer:
What exactly did we test, which build did we test, what risks remain, and who accepted those risks?
There's a detailed discussion of these evaluation criteria in Release Sign-Off Needs More Than a Test List: How to Evaluate Traceability, Approvals, and Jira Sync.
Not every application needs enterprise-level governance.
But every team benefits from knowing what its green build actually proves.
What I would change in a typical automation suite
If I were reviewing a browser automation project today, I'd start with the workflows where users can lose money, data, or progress.
That usually includes checkout, account registration, form submissions, document editing, and any feature involving background synchronization.
For each critical workflow, I'd ask five questions:
- Are we testing state transitions? Not just whether the final button or success message appears.
- Are we verifying persistence? Does the result survive a reload, a new session, or an API read?
- Are failure paths covered? What happens when requests fail, arrive late, or return conflicting data?
- Can we investigate failures quickly? Do we retain enough evidence to understand what happened?
- Can we explain what the tests prove? Are the results connected to requirements and release decisions?
You don't need hundreds of additional tests to make progress here.
Sometimes three carefully designed tests that reproduce realistic race conditions are more valuable than fifty additional happy-path tests.
And that's really the point.
The purpose of test automation isn't to produce an impressive number of green checkmarks.
It's to catch the problems that would otherwise reach users.
A test that proves a Save button can be clicked is useful.
A test that proves the user's work won't silently disappear is considerably more valuable.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.