Latest in AI
llama.cpp 1.0 Upgrade: Fix the defrag-thold Router Error
The new release removes deprecated options and APIs, so an old preset can stop the server before a model loads.
llama.cpp 1.0.0 arrived on November 13 with a practical upgrade trap: a router preset containing defrag-thold can prevent startup. The option has been removed, so troubleshooting should begin with the launch configuration rather than downloading the model again.
The official release includes the merged API cleanup. This guide focuses on that verified change and the checks it creates for existing deployments, rather than treating the version number as a promise that every integration will keep working unchanged.
Why defrag-thold stops the router
The merged cleanup pull request removes -dt, --defrag-thold and the corresponding context-parameter field. A reviewer identifies the specific router failure: a preset retaining the key can produce option 'defrag-thold' not recognized and stop the whole router.
That is different from a model failing to fit in memory. If the process rejects its configuration, changing quantization or GPU layers is addressing a later stage that it has not reached.
Search the actual startup command, service definition and INI file passed to --models-preset. Remove the obsolete option and its value from the command, or the defrag-thold entry from the preset. The reviewer notes that this setting was already a no-op, so replacing it with a guessed tuning flag is unnecessary.
Make a copy of the original configuration first. If several unrelated settings have changed during the upgrade, a small reversible edit is much easier to diagnose than a freshly rewritten deployment.
Check which configuration the server reads
The version-pinned server documentation describes --models-preset as an INI file. Preset keys can use command-line option names without their leading dashes, and the router passes arguments and environment variables to model instances.
Do not stop after fixing a file that happens to have the right name. Check the service's working directory and the precise preset path it passes. A copied configuration in a development folder will not repair the file used by a running service.
After the edit, restart in a controlled environment and read the first failure in the logs. If the unknown-option error disappears but model loading then fails, that is a new diagnostic stage. Keep the two failures separate rather than concluding that the configuration change did nothing.
Library bindings need a different migration
The same cleanup removes llama_set_warmup and functions marked deprecated in llama.h. It also renames llama_set_embd_state_src to llama_set_ctx_state_src as part of promoting speculative-decoding APIs.
For a native binding or application that no longer compiles, use the deprecation notices in its pre-1.0 headers to identify replacements. Removing a server flag will not fix a program still calling a deleted function. Rebuild and test the wrapper against the headers and library it will actually load, not merely the headers in the latest checkout.
A model file, a server configuration and a native binding are three separate upgrade surfaces. A failure in one is not evidence that all three need replacing.
Keep release and nightly builds distinct
The project's versioning documentation distinguishes semantic release tags such as v1.0.0 from development tags beginning with b. Release tags align the bundled ggml copy with a released ggml version for downstream compatibility.
Record the exact build you are replacing and the one you deploy. “Latest llama.cpp” is not a reproducible environment, particularly when an application package, a system library and a manually downloaded server may come from different distribution paths.
Verify one complete request before restoring traffic
Use the same known model and a short, saved prompt for the first check. Confirm startup, successful model loading and a valid completion through the client your application uses. For a multi-model router, repeat that check with each model name your application routes to. These are recommended acceptance checks, not tests performed for this article.
If the deployment also enables server tools, review authentication before making it reachable on a network. Version 1.0 documents a disabled-by-default --allow-unauthenticated-tools override for non-loopback access. Do not use that override as a generic way to make an upgrade error disappear.
The safest migration changes one variable at a time. Get the old workload working on the new runtime before adding a different model, a larger context or broader tool access.
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help technical teams explain developer tools, migrations and product updates through clear technical writing.