# Trajectory Continual Grading and GS1: Learning From Real Agent Work

Two research previews tackle the grading and replay problems that keep production experience from becoming useful training data.

By Paras Madan | 2026-11-03


Trajectory introduced Continual Grading and Group Size 1, or GS1, on November 2. The research previews address two practical problems in training enterprise agents: deciding whether their work was good and learning from tasks that cannot be replayed. The useful question is how much expert judgment the training process still needs.

## What Continual Grading does

In legal analysis or customer support, a successful answer often depends on standards that a generic AI judge does not know. Writing a separate grading rubric for every training task makes those standards expensive to capture.

[Continual Grading](https://www.trajectory.ai/field-notes/continual-grading) learns a metagrader from a small collection of expert-labeled examples. That metagrader can generate criteria for unfamiliar tasks, rather than asking an expert to write each rubric manually. Making the criteria explicit also gives experts something they can inspect and correct.

Trajectory reports that, on Harvey-LAB, training with generated rubrics matched training with expert rubrics using 48 labeled construction tasks and another 48 for selection. These are the company's experiments, not independent validation or a guarantee for a customer's legal workflow.

There is an important limitation. Searching for a better grader can overfit the small set of examples used to choose it. Trajectory's own experiments show that improved selection-set scores do not consistently translate into better held-out results. **A convincing grading demo is not enough to establish a reliable training signal.**

## Why GS1 uses one attempt per task

Some reinforcement-learning methods compare several attempts at the same task. That is manageable in a resettable test environment, but a live customer interaction cannot always be repeated under identical conditions.

[GS1](https://www.trajectory.ai/field-notes/you-only-rollout-once) learns from one rollout, meaning one attempt, per task without requiring a separately trained critic. It uses information from similar tasks to estimate the expected reward, helping distinguish a good attempt at a difficult task from an ordinary result on an easy one.

In Trajectory's tests, GS1 reached a 68% held-out pass rate on LABBench2, compared with 65% late in training for FlashREINFORCE. On AutomationBench at the tuned learning rate, GS1 scored 21.0%, FlashREINFORCE 19.5% and GRPO 21.4%. The comparison supports a narrower conclusion than universal superiority: single-attempt learning can be competitive, while stability depends on the task mix and training settings.

GS1's usefulness also depends on whether its similarity features actually predict task difficulty. A task description may not contain enough information to do that well.

## Availability and the deployment boundary

Both methods are research previews. Trajectory's [documentation](https://docs.trajectory.ai/introduction) labels learning from production as a private beta, and its [website](https://www.trajectory.ai/) still directs prospective users to a waitlist. This should not be read as unrestricted general availability.

The documented workflow collects traces, trains a candidate, evaluates it against a specification and then promotes the chosen model. The website also describes evaluation and approval before updates reach production. Learning from use therefore does not mean that every customer interaction immediately changes the live agent.

## What this changes for agent teams

The opportunity is to make operational feedback usable without rebuilding every customer workflow in a simulator. But the evaluation boundary remains essential: a team needs evidence that a candidate improved on work outside the examples used to train or select it.

For a founder, the next useful question is less whether an agent can keep learning and more whether the team can reliably tell when that learning made it worse.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders turn launches and research updates into clear stories that developers can understand and share.

[Book a 30-minute call](https://cal.com/paras-madan/30min).