Strata DeepSeek Experiment: Faster Prompt Loading With Two Quality Tradeoffs

A new community Strata branch targets the long wait before a local DeepSeek V4.1 Flash coding session starts. It combines cached-conversation reuse with routing prompt tokens through experts already in memory. The code is available for experimentation, but both techniques change the computation and need stronger quality testing.

What the Strata branch changes

Developer PhialsBasement described the work in two November 1 proposals and continued updating the experimental branch on November 2. This is a contributor fork, not an announced stable Strata release or an official DeepSeek update.

The problem is familiar to local coding-agent users: each session can send a large system prompt before the first useful answer. A changed project path near the beginning can prevent ordinary prefix caching from reusing the otherwise identical material that follows.

The KV-splice proposal tries to reuse matching conversation state past those changes. It removes obsolete spans and processes new lines later. In the author's OpenCode test, roughly 37,500 prompt tokens took 100.3 seconds without the patch and 26 seconds with the splice.

That result came from a specific setup: an RX 9070 XT with 16 GB VRAM, a Ryzen 5800X3D, 64 GB DDR4 and a heavily quantized DeepSeek V4.1 Flash model. It is an author-reported comparison, not a general performance guarantee.

Resident-expert routing trades computation for less disk access

The second technique addresses mixture-of-experts models that do not fit entirely in RAM and VRAM. Fetching selected expert weights from storage can dominate prompt processing even when the GPU has spare capacity.

The resident-routing proposal biases prompt processing toward experts already in memory. Generated reply tokens retain the model's normal routing. With both techniques, the author reports a new OpenCode session processing its remaining 513 tokens in 5.1 seconds while reusing 36,949.

That is time spent processing a mostly reused prompt, not a twentyfold increase in answer-generation speed. Comparing it with a full uncached prompt without explaining the reused work would hide the mechanism that produced the improvement.

The useful product question is whether a developer can get back to work sooner after opening a project. Tokens per second during the final answer measure a different part of that experience.

Why the quality caveat matters

KV splicing is approximate because reused states were originally computed with text that has since been removed. Moving new lines also changes their position relative to the original prompt. A model may still receive the updated directory or date while retaining internal state influenced by the earlier context.

Resident routing introduces a separate approximation: a prompt token can be processed by substitute experts instead of the experts the unmodified model would select. The author explicitly reports changed answers. A small coding test scored 12 out of 13 under both configurations, but different problems failed, with only one run per configuration.

That does not establish equivalent reliability. The report also lists missing repeated benchmarks, broader long-context checks and testing on CUDA, SYCL and Metal. Its tested build uses HIP on AMD hardware.

Who should try this experiment

This is interesting for developers investigating memory-constrained local agents, particularly those whose workflow repeatedly sends large, mostly shared prompts. It is too early to treat it as a drop-in performance upgrade for work where preserving model behavior matters.

A useful evaluation would separate the two changes, compare repeated runs on real project tasks, and check whether project-specific instructions survive session switches. Measuring the combined patch alone cannot tell a team which approximation caused a regression.

The broader lesson is that local-agent latency has several parts. Avoiding repeated prompt work can matter as much as faster generation, but an approximation should earn its place by preserving the behavior the workflow needs.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders turn engineering decisions and product updates into clear stories that developers can understand and share.

Book a 30-minute call.