Building with the Web Models API
Web Models API: Practical Guide In the last article, I focused on why I think an API that exposes open-weight, on-device models to web apps is desperately needed, and why the Web Models API could be a good solution. I
Web Models API: Practical Guide
In the last article, I focused on why I think an API that exposes open-weight, on-device models to web apps is desperately needed, and why the Web Models API could be a good solution.
In this article, I want to get down to brass tacks and show what the code would look like. I'll focus on the developer's side. I'll write more about how the browser implementation works in a future article.
These examples follow the current proposal. The API is still being developed and may change.
The API
Before getting into the examples, here's the API at a glance:
// Request access to a model.
navigator.models.requestAccess(model);
// Generate text, stream a response, or create embeddings.
navigator.models.generate(model, input);
navigator.models.stream(model, input);
navigator.models.embed(model, input);
// Bind a model for repeated calls.
navigator.models.session(model);
// Read model details after access has been granted.
navigator.models.info(model);
navigator.models.list();
For a more extensive look at the input and output styles for the api view the reference.
Use cases
The three examples: summarizing a page, creating embeddings, and drafting image descriptions.
I'll reuse this small helper to check support and request access:
async function requestModel(model) {
if (!("models" in navigator)) {
throw new Error("This browser does not support Web Models.");
}
const access = await navigator.models.requestAccess(model);
if (access.status !== "granted") {
throw new Error("Model access was not granted.");
}
}
}
Summarize pageΒ content
Let's add a summary button to a page that already contains an <article> element.
<button id="summarize" type="button">Summarize this article</button>
<p id="summary" role="status"></p>
const summarizeButton = document.querySelector("#summarize");
const summaryOutput = document.querySelector("#summary");
const textModel = "<text-model-id>";
summarizeButton.addEventListener("click", async () => {
summarizeButton.disabled = true;
summaryOutput.textContent = "";
try {
await requestModel(textModel);
const article = document.querySelector("article")?.innerText;
if (!article?.trim()) throw new Error("No article text was found.");
const result = await navigator.models.generate(textModel, {
messages: [
{ role: "system", content: "Summarize the supplied article in three short sentences." },
{ role: "user", content: article }
],
maxOutputTokens: 200
});
if (result.finishReason !== "stop") {
throw new Error("The summary did not finish. Try a shorter input or a larger output limit.");
}
summaryOutput.textContent = result.data ?? "";
} catch (error) {
summaryOutput.textContent = error.message;
} finally {
summarizeButton.disabled = false;
}
});
The app reads its own page and passes the text to the model. The answer is in result.data. finishReason tells us whether it finished or reached a limit.
Create embeddings
Embeddings let us compare text by meaning. A search for "when is the launch?" could find a note that says "the release moved to October."
Here's how to turn a few notes into vectors:
<button id="index-notes" type="button">Create note embeddings</button>
<p id="index-status" role="status"></p>
const indexButton = document.querySelector("#index-notes");
const indexStatus = document.querySelector("#index-status");
const embeddingModel = "<embedding-model-id>";
const notes = [
{ id: 1, text: "The release moved to October." },
{ id: 2, text: "The design review is on Tuesday." },
{ id: 3, text: "Support hours are 9 AM to 5 PM." }
];
let noteIndex;
indexButton.addEventListener("click", async () => {
indexButton.disabled = true;
indexStatus.textContent = "";
try {
await requestModel(embeddingModel);
const result = await navigator.models.embed(
embeddingModel,
notes.map(note => note.text)
);
noteIndex = {
model: result.model,
entries: notes.map((note, i) => ({
...note,
vector: result.vectors[i]
}))
};
console.log(noteIndex);
indexStatus.textContent = `Created embeddings for ${notes.length} notes.`;
} catch (error) {
indexStatus.textContent = error.message;
} finally {
indexButton.disabled = false;
}
});
embed() gives us a vector for each note. I've kept the model ID with them because we'll need the same model for search queries.
Accessibility tools
Let's have the model write a first draft of alt text for an image. We'll put it in a text field so the author can edit it before publishing.
I'm asking for a structured response here, so the model needs to support image input and JSON Schema output.
<label for="image-file">Choose an image</label>
<input id="image-file" type="file" accept="image/*">
<label for="image-context">How will this image be used?</label>
<input id="image-context" type="text">
<button id="describe-image" type="button">Draft alt text</button>
<label for="alt-draft">Alt text - review before publishing</label>
<textarea id="alt-draft"></textarea>
<p id="image-status" role="status"></p>
const describeButton = document.querySelector("#describe-image");
const altDraft = document.querySelector("#alt-draft");
const imageStatus = document.querySelector("#image-status");
const visionModel = "<vision-model-id>";
describeButton.addEventListener("click", async () => {
describeButton.disabled = true;
altDraft.value = "";
imageStatus.textContent = "";
try {
const image = document.querySelector("#image-file").files[0];
const context = document.querySelector("#image-context").value;
if (!image) throw new Error("Choose an image first.");
if (!context.trim()) throw new Error("Describe how the image will be used.");
await requestModel(visionModel);
const result = await navigator.models.generate(visionModel, {
messages: [{
role: "user",
content: [
{ type: "text", text: `Draft concise alt text for this image. Avoid guessing details. Page context: ${context}` },
{ type: "image", image }
]
}],
output: {
schema: {
type: "object",
properties: { alt: { type: "string" } },
required: ["alt"],
additionalProperties: false
}
},
maxOutputTokens: 120
});
if (result.finishReason !== "stop") {
throw new Error("The description did not finish. Try again with a larger output limit.");
}
altDraft.value = result.data.alt;
imageStatus.textContent = "Draft ready for review.";
} catch (error) {
imageStatus.textContent = error.message;
} finally {
describeButton.disabled = false;
}
});
The image comes straight from the file picker. Asking for an object with an alt field gives us result.data.alt, with no JSON string to parse.
Other ideas
- Rewrite selected text in simpler language.
- Extract fields from a receipt into a structured object.
- Receipt organizer.
- Choose a UI component based on generated content.
- Use tool calls to filter a list or update a form, with the app checking and running each call.
- Transcribe audio clips.
- Offline privacy focused text-to-sql data explorer.
Final Thoughts
With this API, I could add these features to a web app without managing model downloads and execution in every project. This is still very much a proposal, but one that I believe is badly needed. If you're interested in helping move this forward, start by reading the explainer and API reference and joining the WICG discussion.
Web Models API β Models belong in every browser.
A proposed browser API for web apps that need a particular open-weight model. People control access; the browser runs it. Join the Web Models API discussion.
webmodels.dev
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.