Sakana MASS: Teaching a Model From Its Own Agent Teams

Sakana AI has introduced MASS, a training method that lets one model learn from the work of its own agent teams. Its October 12 announcement reports better research performance per output token after two training cycles, offering a concrete experiment in recursive self-improvement with an open-weights model.

MASS stands for Multi-Agent Self-Supervision. Developed with UC Berkeley researchers, it uses the same model to perform research, improve how agents collaborate and judge their results. Team executions ranked highest by the model become training material for its next version.

This is useful to examine because an agent's results depend on decisions around the task as well as the answer itself. Who checks the data? What does the next agent receive? When should a weak result trigger another attempt? MASS tries to make some of that coordination learnable.

How Sakana MASS works

The paper, submitted on October 8, starts with Qwen3.6-27B running inside qwen-code. One loop searches for better workflows while keeping the model's weights fixed. Another fine-tunes the model on selected conversations from those workflows, including both orchestration and individual agents' work.

The updated model then takes over all three jobs again: executor, optimizer and evaluator. The researchers completed two cycles. That gives them evidence about improvement across two updates, while leaving the question of sustained improvement over many generations open.

For evaluation, the researchers used GPT-5.5 and Claude Opus 4.8 separately from the training loop. Those judgments were withheld from MASS. On three synthetic research tasks held out from post-training, the second-round model received 69.9% of non-tied judgments in comparisons against the original model. That is a judged comparison on a small test set, so its scope matters.

Agent handoffs become training material

The detailed research explainer makes the workflow fairly tangible. Each agent has a role, instructions, an output contract and a place in the sequence of calls. The contract describes what it must return and which checks it must complete.

In a finance example, the workflow asks for checks that historical training, validation and test periods remain properly separated. Later agents receive evidence about those checks. This matters because a backtest can appear successful when it accidentally uses information unavailable at the time of a decision.

For an agent builder, that suggests a practical place to investigate failures. If a downstream agent repeatedly makes the wrong decision, inspect the evidence passed to it. Giving it a more elaborate role description may be less useful than requiring the previous step to return a missing check or an explicit uncertainty.

What the benchmark gains actually measure

Sakana reports score per output token reaching 1.2 to 1.6 times the baseline across MLR-Bench, DSBench, ScienceAgentBench and AstaBench's E2E-Bench-Hard tasks after two cycles. The headline measures benchmark score relative to generated tokens. It does not establish the same improvement in response speed, API spending or the total compute used to search, generate training data and train the model.

The repository's results show why the underlying scores deserve attention too. ScienceAgentBench rises from 28.8% to 31.7%, while DSBench moves from 0.473 to 0.482. Improvements in the efficiency ratio can therefore look larger than changes in the task score alone.

Transfer also varies by task. Terminal-Bench 2.0 falls from a 37.5% mean pass rate to 35.2%, and SWE-bench Verified falls from 62.6% to 61.1%. A team considering this method for software engineering would need its own evaluation before assuming the research-task gains carry over.

The release has reproduction limits

The code is available, but its implementation notes explicitly describe differences from the reported experiments. The current search driver uses a fixed reference in a way that differs from the paper's algorithm. The complete historical run ledger and exact second-cycle data and budget manifest are also absent.

The package's offline tests do not reproduce the research scores, and end-to-end benchmark reruns have not been validated for the package. An available implementation and a reproduced result are separate milestones. Anyone allocating training resources should read those notes before treating the repository as a ready-made replication recipe.

There is also a more basic limitation in the learning loop. A model that creates and judges its own work can preserve the same mistakes through both steps. Sakana explicitly raises this shared-blind-spot problem. Better internal agreement alone cannot settle whether the resulting work is correct.

The useful next test is whether independently checked gains survive more rounds, broader tasks and complete accounting of the work needed to obtain them. For developers, that means keeping failure cases and external checks alongside the successful traces selected for training. Otherwise, an improvement process can gradually lose the evidence needed to question its own results.

I'm Paras Madan. I work at the intersection of AI engineering and distribution, helping people understand and adopt technical products.

If you're launching an AI product or research tool, I help with technical writing, launch storytelling and founder-led distribution that make the work and its limits clear.

Book a 30-minute call