Latest in AI
How to test fallback model tracking in Qwen Code 0.28.3
Try Qwen Code's opt-in model availability tracking and verify when a fallback is skipped, when it is tried again, and why the primary still runs.
A coding agent can make several model requests while answering one prompt. It might ask for a file, read it, and then call the model again with the result. If each request retries the same overloaded fallback before reaching a healthy one, a short task can repeatedly pay the same wait.
Qwen Code 0.28.3, released November 21, adds experimental model availability tracking for this case. The important detail is that it remembers exhausted fallbacks, not that it switches the whole session away from the primary.
Decide if this is the delay you actually have
Start with a workflow that already has a fallback chain. Single-model users are not the audience for this switch. Neither are teams trying to solve chronic latency on a healthy primary or a permanent billing quota problem.
The implementation notes describe the behavior after a fallback has used up its retry budget on an eligible capacity error. In the rest of the same turn, Qwen skips that fallback when it reaches the chain again. The primary is still attempted on each request. If a primary is the one consuming most of your time, this change may not noticeably help.
Enable it in one test project
The versioned settings reference lists experimental.modelAvailabilityTracking as off by default and requiring a restart. In a disposable workspace, merge the following into the project's .qwen/settings.json, preserving any existing settings:
{
"experimental": {
"modelAvailabilityTracking": true
}
}
Restart the session before testing. Keep the primary and fallback choices the same in both runs so the setting is the only variable. Test a normal interactive session first, rather than assuming the result applies to background requests or a different client surface.
Compare requests within one turn
Use a small synthetic code fixture and a read-only task that needs the model to inspect a file and then explain it. The goal is to distinguish the model requests before and after the file read, not to judge the explanation.
Use the project's existing local test harness or controlled provider stub to supply failure and success outcomes. Do not send extra traffic to a real service to induce throttling. Make the primary fail with a modeled capacity error, have fallback A exhaust its retry budget, and let fallback B answer. For the next model request in the same turn, compare the attempt sequence with tracking off and on.
With tracking on, the primary remains in the sequence, but fallback A should not get another retry cycle in that turn. Keep a trace of the model label, attempt order and elapsed time, not the prompt or repository contents. A successful final response alone does not show that the extra attempts went away.
Check recovery instead of assuming a blacklist
Use a new top-level prompt after letting fallback A recover. Arrange the fixture so the primary still cannot serve that request. Qwen should be willing to try the recovered fallback again. The release's design also re-probes on an explicit retry. That makes the setting a short-term response to capacity pressure rather than a permanent exclusion list.
Add a separate permanent-quota case. The settings reference says permanent quota exhaustion still fails fast. Do not classify every 429 as transient just because the status number can appear in the capacity-error path.
Keep the rollout small
Compare latency and attempt counts on the same fixture across several runs. If your actual workload seldom reaches a second fallback request in one turn, there may be little to gain. If it regularly retries the same exhausted fallback, start with one team or project and document what changing the switch can and cannot fix.
Explain runtime changes in plain language
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help AI and developer tool teams turn releases and configuration changes into technical writing and product stories developers can use.