PARAS MADAN
All writing

Latest in AI

Edge Prompt API Origin Trial: Add Local AI Without a Silent Cloud Fallback

Plan an Aion-powered browser feature around origin-trial tokens, device eligibility, model downloads and an explicit choice before sending text to the cloud.

A local AI demo working on your laptop does not tell you whether it will work for someone visiting your website. Their browser may expose no API, their device may be ineligible, or the model may still need downloading.

Microsoft's new Edge Prompt API origin trial makes those product questions worth testing on a live site. The useful starting point is not a universal chatbot. It is a small optional feature with clear states and a sensible experience when local inference is unavailable.

Consider a support form that can turn a user's rough notes into a clearer draft. The person still reviews and submits the message. That keeps the local assistant separate from the final action.

Separate the model launch from API availability

The November 19 announcement contains three different release stages: Aion Instruct is generally available as open weights and Edge's built-in model; the Prompt API gets an origin trial; the Decisions API remains an experimental developer preview in Canary and Dev.

Aion replaces Phi-4-mini in Edge. That does not make the Prompt API a universally available browser feature, nor does it mean the Decisions API is included in the same live-site trial.

This is also a separate integration from the native Windows preview. A web team is evaluating browser access and visitor experience, not distributing Windows runtime packages. Keep those release decisions separate even though the model name is shared.

Register the right origin and plan for expiry

Microsoft's origin-trial instructions explain the delivery mechanism: register the site's origin, then supply the issued token through a meta element or an Origin-Trial response header. Eligible visitors do not need to change browser flags themselves.

Registration distinguishes a domain, a particular subdomain, and optional subdomain coverage. Paths do not narrow the registration. Use an HTTPS test origin that matches your intended scope rather than assuming a token for one hostname covers every deployment.

Check the active trial's supported browser versions and expiry during registration. Search results can also surface older trials with similar names.

Assign someone to token renewal and keep the feature optional. Trials can end early, and an expired token must not stop a customer from submitting a support request.

Treat eligibility as a product state

The current Edge Prompt API documentation lists Windows 10/11, macOS 13.3+, and x64 Linux with Vulkan drivers and Edge 159+. Requirements include at least 3 GB VRAM, 20 GB free on the Edge-profile volume, and an unmetered connection. Below 5 GB free, Edge deletes the model.

Enterprise policy or the On-device AI setting can block availability. Canary, Dev and Beta have that setting off by default unless the relevant local-testing flag enables it.

Those are eligibility constraints, not a model-download size. Do not tell visitors they must download 20 GB. More importantly, do not turn an unsupported machine into an error that asks the user to fix their computer. Keep the normal text editor working and explain that the optional local assistant is unavailable.

Design the download and generation states

Check for LanguageModel, then use LanguageModel.availability(): unavailable, downloadable, downloading and available are distinct outcomes. The session's monitor option exposes download progress. Ordinary generated text also carries an invisible watermark from Edge 158; constrained responses are exempt.

For the support form, I would show a separate “Prepare local assistant” action before generating anything. Explain that first use may require a download, preserve the user's notes, and let them continue without AI. A progress display should not lock the whole form.

The Prompt API draft defines session creation, prompting, streaming and cancellation signals, but explicitly is not a W3C Standard. Implement against Edge's documented behavior and test other browsers separately rather than assuming every draft option ships everywhere.

My acceptance checklist would include cancelling generation, navigating away mid-download, returning with the model available, and submitting the original notes without accepting a rewrite.

Make cloud fallback an explicit choice

Microsoft describes hybrid intelligence as developers combining local models with a configured cloud provider. That is an architectural option, not permission to upload a visitor's text automatically when the local route fails. Edge also lets users disable on-device AI, removing the model and stopping future downloads.

For this form, offer “Continue without AI” first. If a cloud alternative exists, name the provider, explain which text would leave the device, and ask before sending it. Keep that choice separate from submitting the final support request.

Evaluate whether the rewrite preserves facts, not just whether it sounds smoother. A polished draft that changes an order number or invents a promised refund is worse than the rough notes. Keep the original visible and require review before submission.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help developer-tool teams explain new APIs and practical product tradeoffs through clear technical writing and storytelling.

Book a 30-minute call.