DecisionFormer: Train and Serve Open Decision Models Without Text Generation

FrontiersMind announced DecisionFormer on October 26, an open-source framework for fine-tuning and serving models that choose between defined options without generating a text answer. It is aimed at tasks such as routing requests and applying yes-or-no gates, where developers need a structured decision rather than a conversation.

The team’s announcement introduces the training code behind its decision-model work, alongside the deciformer-15k dataset. The framework supports full fine-tuning, LoRA and QLoRA, so teams can adapt a backbone instead of building a separate training pipeline for each approach.

What a decision model returns

DecisionFormer takes a state, such as a support request, and typed questions about it. A choice question returns probabilities over named options. A yes-or-no question returns the probability of yes. An ordered score uses a defined set of levels.

For a routing task, the application can provide its available queues rather than ask a chatbot to invent a department name. The model scores the permitted answers in a forward pass instead of generating text that the application must parse.

A constrained answer format removes one integration problem, not the possibility of a wrong decision. A model can return a perfectly valid queue name and still send the request to the wrong team.

That is why the useful comparison is against the workflow the application already has. Measure routing quality, uncertain cases and processing time on representative requests, including the ones people usually correct by hand.

Installation, API and availability

The public repository uses the Apache 2.0 licence. It documents Python 3.10 or newer, PyTorch 2.6 or newer and Transformers 5.20 or newer as requirements. The serving extra adds an HTTP API at /v1/systemone.

The current PyPI package is version 0.2.0 and is labelled alpha. Its release history dates that package to October 23 UTC, before the October 26 announcement. The announcement should therefore not be read as the release date of a new package version.

The documented installation is:

pip install "decisionformer[serve]"

This installs the framework and serving dependencies, rather than granting access to a hosted inference service. Developers still need an appropriate model and an environment to run it. The library’s licence also does not replace the licence conditions of the model weights they choose.

Why confidence needs a closer look

The project distinguishes an option’s probability from its confidence field. For its default format, confidence measures how concentrated the answer distribution is using normalized entropy. A distribution split almost evenly between two options has low confidence even though one option must rank first.

The training workflow also fits a calibration temperature using held-out data. Calibration attempts to make probabilities better reflect observed outcomes, rather than merely make the winning answer look more certain.

For a product team, this creates room for a deliberate fallback. An ambiguous request could go to a human or a more capable model instead of being forced through the cheapest route. The threshold should come from the cost of mistakes in that application, not from assuming that a field called confidence means the same thing across every model.

Where DecisionFormer still has limits

The repository does not support Hub models that require custom trust_remote_code. It also lists image and video inputs as roadmap work. Its broad backbone support should not be confused with support for every model or modality.

The maintainers warn against assuming that changing a causally trained decoder to bidirectional attention will improve it. That makes a controlled baseline important: change one part of the recipe, evaluate it, and retain the change only if it helps on the relevant data.

The useful opportunity is a smaller, testable decision step inside a larger application. If the answer space is known, teams can separate that step from general-purpose generation. The next question is whether the resulting model earns its place through better reliability, latency or operating cost on the actual workload.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help teams turn technical releases and developer tools into clear stories that explain where a product fits and where it does not.

Book a 30-minute call.