# oMLX 0.7.3 Adds Voice, Web Search and Screen Capture to Local AI

The Mac inference app is becoming a more useful desktop assistant, but the model and permissions still decide what it can actually do.

By Paras Madan | 2026-11-02


oMLX released version 0.7.3 on October 31, adding voice conversations, web search, link reading and screen capture to its Ask AI interface. For people running models on a Mac, the useful change is having fewer steps between a question and the context needed to answer it.

The [release](https://github.com/jundot/omlx/releases/tag/v0.7.3) also improves inference performance and memory use. But the desktop workflow is the more accessible reason to look at this update, even if you are not serving several large models at once.

## What oMLX Ask AI can do now

Ask AI already had a question bar accessible from other apps. Version 0.7.3 adds a voice mode: enable Voice under App > Ask AI, press Option-Shift-Space and speak. Responses play back with captions. The assistant can also use tools to find information online, read a URL or inspect a screenshot.

That makes a practical difference to a familiar task. If you are looking at an error message, you can ask about the screen rather than manually taking a screenshot, finding a chat window and attaching it. If the answer depends on current documentation, web tools can bring that material into the conversation.

These are context-gathering features. Their usefulness still depends on whether the selected model understands the material and gives a reliable answer. A smoother interface does not establish that the answer is correct.

## Voice and screen capture have different requirements

The [Ask AI settings implementation](https://github.com/jundot/omlx/blob/v0.7.3/apps/omlx-mac/Sources/AppView/Screens/AskAIScreen.swift) makes an important distinction: the voice model picker lists models with a realtime voice backend. Installing an ordinary text model does not automatically give it this capability.

Screen capture needs an image-capable model and macOS screen-recording permission. The interface disables the option for models that cannot read images. Microphone access is separately required for voice.

Web search uses the provider configured in Dashboard Settings > Integrations. **Local inference does not make web tools offline.** Treat the model, search provider and screen permissions as separate decisions, especially when working with customer material or internal documents.

This is worth checking before turning on every option. A screenshot can contain far more information than the short question you intended to ask, so inspect what is visible and enable only the capabilities your workflow needs.

## Installation, API access and cost

The [installation guide](https://github.com/jundot/omlx/blob/v0.7.3/README.md) specifies Apple Silicon and macOS 15 or later. The project is Apache-2.0 licensed, with a downloadable Mac app, Homebrew installation and source installation options. Model licenses remain a separate consideration.

The official app bundles the native kernels. Building those kernels from source requires full Xcode with the Metal toolchain, rather than Command Line Tools alone. For someone who wants to use the assistant rather than maintain a build environment, the packaged app is the simpler starting point.

Existing OpenAI-compatible clients can connect to the local server at `http://localhost:8000/v1`. This release also adds `/v1/realtime`, but describes it as supporting a **subset** of the OpenAI Realtime API. Developers should check the specific events and behavior their client requires rather than assume complete compatibility.

Running a local model avoids treating every inference call as a hosted API purchase. It does not remove hardware costs, model download requirements or any charges attached to the search provider you configure.

## Read the performance numbers as workload-specific

One reported test reduced peak memory from 248.0 to 161.1 GiB with eight long-context Qwen3.8-Flash-Next streams on an M3 Ultra. The release attributes lower concurrent-memory use to updating the batch KV cache in place rather than copying it.

Those are project-reported results, not independent measurements. They describe a demanding multi-request workload, not the memory needed for a small assistant on a laptop. They also should not be treated as a prediction for a different model, context length or Mac.

For a developer serving several agent conversations, peak memory can matter more than the fastest single response. For someone asking occasional desktop questions, model quality, response delay and how easily context reaches the model are likely to matter more.

The useful test for this update is therefore fairly ordinary: can a local assistant help with the work already on your screen without requiring a separate ritual to provide context? That is a more meaningful adoption test than another impressive tokens-per-second number.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders turn launches and product updates into clear stories that developers and founders want to read and share.

[Book a 30-minute call](https://cal.com/paras-madan/30min).