# Oumi Expands FAiE Agent Training: Bring Your Own Harness and GRPO Preview

Oumi's November launch connects agent evaluations to model training. Check Bring Your Own Harness requirements, GRPO's private preview and usage-based costs.

By Paras Madan | 2026-11-05


Oumi's [November 4 fall launch](https://blog.oumi.ai/events/fall26) expands its AI engineering workflow around agents and the models they use. Teams can build an agent, evaluate its actions, generate training data and train the underlying model, or bring an existing agent harness into the process.

The launch includes direct preference optimization alongside supervised fine-tuning and on-policy distillation. Reinforcement learning through GRPO is in private preview. Teams evaluating the product should distinguish that restricted access from capabilities they can use immediately.

The useful question is whether a failure calls for better model behavior. An incorrect tool argument might be trainable; a missing permission or broken integration usually needs a different fix. Collecting traces and evaluating failures gives a team a way to make that decision before spending on training.

## Keep the harness, connect it to Oumi

[Bring Your Own Harness](https://docs.oumi.ai/guides/harnesses) lets an existing agent retain its prompts, tools, memory and state in its own runtime. It requires a public HTTPS endpoint implementing Oumi's harness protocol.

During a run, Oumi sends conversation inputs and supplies a scoped connection to the model. The customer's harness executes its own tools and returns messages. Oumi evaluates the resulting conversations or saves them as a synthesized dataset. Custom Oumi models currently support harness evaluations, but not harness synthesis; supported API-hosted models can do both.

This is an integration project, even if the team keeps its current orchestration. Review what the endpoint exposes and which tools an evaluation can reach. Start with isolated test resources so a synthetic conversation cannot accidentally change customer records or trigger a real transaction.

It is also worth preserving the current harness version with each evaluation. Otherwise, a changed prompt or tool implementation can make a model comparison difficult to interpret.

## Pick the training method after the evaluation

Oumi's [training documentation](https://docs.oumi.ai/guides/training) describes DPO as learning from preferred and rejected responses to the same prompt. SFT uses examples of the desired response. On-policy distillation draws on a teacher model's judgment of responses produced during training.

Methods can be combined across runs, but Oumi warns that training on top of a trained model can forget earlier behavior. It recommends checking the full evaluation set, not just the failures targeted by the latest run. Deployment support also depends on the chosen base model and fine-tuning method.

For a pilot, choose one repeated failure with a measurable outcome. Reserve examples that never enter the training data and compare the trained model against the existing system. Include cases the old model already handled well. A higher score on the new examples is less useful if routine tasks get worse.

## A model endpoint still needs an agent runtime

The [platform guide](https://docs.oumi.ai/guides/intro) says FAiE can create evaluators and datasets, run training and help manage deployments. It asks before actions with costs or side effects unless Auto-approve is enabled. Saved recipes let teams inspect and rerun job configurations.

The guide also makes a deployment boundary explicit: a model endpoint does not itself execute an agent's tools or orchestration. Those pieces still need a runtime and an integration with the deployed model.

## Budget for the experiment, not just the plan

Oumi's [pricing page](https://oumi.ai/pricing) lists a free tier with up to $50 in credits for three months. Pro costs $25 per month, includes $25 in monthly credits and continues on usage-based billing. Pro lists one model-weight download per month; Enterprise pricing is custom.

Agent tokens, evaluation, data generation, training and production inference have separate charges. Dedicated production GPUs are billed for uptime. The pricing page does not clearly establish that every new training method is included in the Pro plan, so confirm the required method and access before budgeting around that subscription price.

A sensible first budget covers a baseline, a limited training run and a fresh evaluation. Decide what improvement would justify another iteration before expanding the experiment.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders turn AI infrastructure and developer-tool updates into clear technical writing and product stories.

[Book a 30-minute call](https://cal.com/paras-madan/30min).