Edge0-145B Local Setup: GPU Memory, Windows Support and License Limits

Edge0 released Edge0-145B and Edge0-120B for local inference, with Linux and Windows build instructions for NVIDIA GPUs using CUDA 13. Before downloading either, check three things: which runtime you need, what the memory numbers actually measure, and whether the model license covers your intended use.

Edge0-145B hardware requirements need careful reading

The official model guide lists Linux or Windows, an NVIDIA GPU, CUDA 13, at least 16 GiB of GPU memory and 20 GiB of system memory. Those are the guide's stated requirements, not an independent hardware recommendation.

There is an important documentation distinction. The repository's measurement notes describe the 16 GiB VRAM plus 20 GiB RAM benchmark as a constrained RTX 5090 D test, and explicitly say that test budget is not a minimum system requirement. The setup guide separately lists the same capacities as requirements. A constrained flagship GPU is not evidence that every 16 GiB card performs the same way.

These are the published 145B weight sizes and memory footprints with residency enabled on the 24 GB RTX 5090 D:

Edge0-145B format	Weight size	Measured GPU memory	Measured system memory
Q3_K	40.81 GiB	23.14 GiB	31.28 GiB
NVFP4	47.79 GiB	23.25 GiB	38.27 GiB

The GPU and system-memory columns contain different expert sets, according to Edge0. They are not two copies of the same allocation. Weight size also does not describe all runtime memory, so adding up the model files is not a reliable capacity plan.

Does Edge0-145B run on a Mac?

The current 145B and 120B build instructions target NVIDIA CUDA on Linux and Windows. Edge0 publishes an Apple M2 Ultra Metal measurement, but explicitly says that this does not establish macOS release availability.

Build the engine and launcher separately

The 145B and 120B models run through edge0-strata. The edge0-infer program launches that engine and provides the client-facing interface. Building the launcher alone is therefore not enough.

Use the official getting-started instructions for the launcher and the linked platform-specific engine build instructions. The commands below begin after both builds and the model download are complete. They are documented setup steps, not a report of a model run performed for this article.

Use the intact Edge0-145B-q3k folder containing model.json, not a parent holding several models.

From the edge0-infer directory on Linux, inspect the assets before loading the model:

export EDGE0_145B=/path/to/Edge0-145B-q3k
build/cpu/bin/edge0-infer inspect --model "$EDGE0_145B" --json

A complete selected configuration produces ok: true. This checks assets, not generation quality or speed.

Start the local server only after the inspection succeeds:

build/cpu/bin/edge0-infer serve --model "$EDGE0_145B"

Wait for the Listening: message before sending a request. The default local endpoint uses port 8000 and the default model alias is edge0_145b:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"edge0_145b","messages":[{"role":"user","content":"Explain this model in two sentences."}],"max_tokens":64}'

On Windows, use build\windows\bin\edge0-infer.exe with a Windows path, not the Linux shell syntax. Keep this first test on loopback, not a publicly exposed endpoint.

Troubleshoot the runtime before changing the model

For these models, use serve, chat or inspect, not the older native engine's run command. If inspection reports missing files, correct the model directory before retrying. Changing a model option does not convert one released expert format into another.

Do not start chat alongside an existing serve process on the same GPU: it launches another engine rather than connecting to the first. A compatible client should connect to the server you already started.

A successful short response establishes a working installation, not production readiness. Before connecting a coding agent, try a small repository task with a known answer, inspect the proposed changes and measure the workload on your actual machine. Keep those observations separate from the publisher's benchmark numbers.

The model license is not the framework license

The framework's Apache-2.0 license does not make every use of the model unrestricted. The Edge0 Community Model License 1.0 separately governs the designated Edge0 model contributions, with upstream Qwen terms remaining applicable.

The Edge0 grant permits personal experimentation, research, internal organizational use and commercial use of outputs, subject to its conditions and upstream terms. It requires a separate written Edge0 license for specified commercial offerings, including paid model distribution, qualifying commercial inference or fine-tuning services, and independent commercial coding or office-assistant products. The last category can apply even when inference runs on the customer's device.

There are explicit distinctions for incidental features and single-purpose domain tools, so do not reduce this to either “commercial use is banned” or “commercial use is free.” If your product falls near those boundaries, review the actual terms and obtain the relevant permissions before committing to the model. Edge0 permission does not waive Qwen obligations.

The first decision is therefore narrower than whether a 145B model can fit on a desktop. Check whether the released runtime supports your machine and whether the license supports your product. Only then does a performance comparison become useful.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help technical teams turn model releases, infrastructure choices and product tradeoffs into clear writing developers can use.

Book a 30-minute call.