Dev.to AI 🤖 Ai 👁 0 📖 5 min read

In-Browser AI Inpainting: How It Works and How to Get Better Results

AI inpainting is the technique behind most "erase this from my photo" tools. You mark a region, and a model invents pixels that look like they belong there: pavement under a trash can, sky behind a pole, wall behind a st

AI inpainting is the technique behind most "erase this from my photo" tools. You mark a region, and a model invents pixels that look like they belong there: pavement under a trash can, sky behind a pole, wall behind a sticker.

A lot of this now happens in the browser. Sometimes the editor is a web app and the model runs on a server. Sometimes the model itself runs on your device, through WebGPU or ONNX Runtime Web. Either way, quality depends less on the brand of the tool and more on how you prepare the image and the mask.

This guide covers that workflow: what the pipeline is doing, how to give it better input, and the mistakes that produce blurry patches or repeating textures.

What inpainting is actually doing

An inpainting pass looks like this:

  1. The image is prepared. Many tools resize the long side to a fixed maximum (often around 1K–2K pixels) so the model fits in memory and finishes in a few seconds.
  2. You provide a mask. The mask is a binary (or soft) map of the pixels to replace. In a web UI, that usually means a brush.
  3. The model looks at the surrounding context. It does not "know" what was behind the object. It predicts a fill that is statistically consistent with nearby texture, edges, and lighting.
  4. The fill is composited back. Only the masked region should change. A before/after slider helps, because small seams are easy to miss at thumbnail size.

Two consequences follow, and they drive the tips below.

First, the model can only use the context you leave it. If a person covers the only view of a patterned tile floor, the fill will be a guess. If you leave enough of that pattern visible, the guess gets much better.

Second, a bad mask is a bad edit. Tight masks leave halos. Masks that miss a shadow or a reflection leave "ghosts." Huge masks force the model to invent a large area with little constraint.

Server-side vs on-device, briefly

If you are choosing or building a tool, the split matters:

  • Server-side inference can use larger models and is consistent across devices. The image has to be uploaded, so you should check the tool's retention policy.
  • On-device inference (WebGPU, WASM, ONNX) keeps the pixels on the machine and can feel instant after the model loads. It is limited by the device: a phone may downscale more aggressively, or fall back to a smaller model.

The mask-and-review habits below apply to either setup. A browser tool such as ClearRemover follows the same shape: upload a photo, brush the area, review, and download.

Step 1: Start with a usable source image

  • Prefer the original file over a screenshot or a heavily compressed social-media download. Compression ringing around edges confuses both the mask and the fill.
  • Avoid images that have already been upscaled or "enhanced." The model may reproduce those fake textures.
  • If the object sits on a detailed background (a brick wall, a wooden floor), keep as much of that background in frame as you can. Tight crops starve the model of context.
  • Shoot or export at a reasonable resolution. A 24 MP original is fine, but a 400 px thumbnail will not suddenly grow clean detail.
  • Check the tool's processing and export caps. Many web tools scale the long edge to around 2048 px for the edit, and some free exports are smaller than the working image. If you need a large result, confirm the download size before you rely on the edit.

Step 2: Build a better mask

  1. Cover the whole object, plus a small margin. A 5–10 pixel margin usually prevents a leftover outline. Going far beyond that just deletes useful context.
  2. Include what the object caused. Shadows, reflections, and items a person is holding belong in the mask. A removed lamp post that still casts a shadow looks like a mistake, not a clean street.
  3. Don't join unrelated objects into one mask unless they overlap. One trash can and a distant sign are two passes. Each pass leaves a cleaner scene for the next.
  4. Use the right category if the tool offers one. A "people" mode may also remove the shadow and carried items. A "text" mode is usually better for date stamps and captions than a generic object brush.

Step 3: Work in short passes

Do the largest or most central distraction first. Check the result. Then do the next one. After a first pass, a second, smaller brush over a blurry patch or a broken line is usually better than undoing everything.

If two objects overlap a complex structure (a fence, a bicycle wheel, lettering on a storefront), consider whether cropping would be cleaner. Inpainting invents pixels; cropping does not. When the subject you want is close to the frame edge and the clutter is at the border, a crop is often the honest fix. Save inpainting for clutter that sits inside the composition.

Step 4: Inspect like a skeptic

Zoom to 100%. Look for:

  • Repeated blobs of texture (a classic inpainting artifact)
  • Lines that almost meet but don't: curbs, tiles, railings, horizons
  • Soft, waxy patches that don't match the grain of the rest of the photo
  • Color shifts at the mask boundary
  • Leftover fragments: a shoe, a sliver of a sign, a reflection in a window

Use a before/after slider if you have one. Toggle it a few times. Your eye adapts quickly to a bad fill if you only stare at the after image.

Common mistakes

  • Masking too tightly. The outline remains, especially with hair, wires, or bike spokes.
  • Masking too loosely on a detailed structure. The model has to rebuild a window frame or typography it can only partly see, and it will often smear it.
  • One giant pass over a busy scene. The first fill becomes context for later mistakes.
  • Ignoring resolution limits. A clean 1024 px export can look soft if you later place it in a large print or a hero banner.
  • Trying to restore what isn't there. A person standing in front of a unique mural gives the model nothing reliable to copy. If you don't have another frame of the same scene, you may need to crop, clone from a second photo, or accept a simplified fill.

A compact checklist

  • Original file, not a compressed screenshot
  • Enough surrounding context in the frame
  • Mask covers the object, its shadow, and a small margin
  • One subject per pass
  • 100% inspection for seams, repeats, and leftover fragments
  • Export size checked against how you'll use the image

Wrapping up

In-browser inpainting is still a constrained prediction: a mask plus nearby pixels in, a plausible fill out. Give the model clean context, mask a little more than the object and a little less than the scene, and review at full size. Those habits matter more than which web tool you open.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.