# Rumble-2-Flash Setup: Control Transcript Formatting Before Comparing Accuracy

Reson8's open speech-recognition model exposes useful formatting controls, but batch throughput and cleaned transcripts need careful interpretation.

By Paras Madan | 2026-11-18


Reson8's November 18 [Rumble-2-Flash release](https://huggingface.co/blog/rumble-2-flash) introduces a 0.5-billion-parameter speech-recognition model supporting more than 40 languages under Apache 2.0. It uses the Qwen3-ASR architecture and works with Transformers and vLLM rather than requiring a separate inference fork.

The useful starting question is not whether its headline throughput looks impressive. It is whether you can produce the kind of transcript your application expects, consistently, on the audio you actually receive.

For a first evaluation, I would settle transcript formatting before comparing accuracy. Otherwise, you risk measuring differences in presentation as though they were recognition mistakes.

## Start with one short, authorized recording

The [model card](https://huggingface.co/Reson8/Rumble-2-Flash) provides a Transformers path using `AutoProcessor` and `AutoModelForMultimodalLM`, and a vLLM serving path. Its processor accepts audio paths, URLs or arrays and can return transcription-only text. The example supplies a generation limit rather than promising unlimited transcription.

Use a short recording you are allowed to process, with a written reference and a known language. Keep the first run small enough that you can listen to every word. Record the model revision, package versions, device, generation settings and elapsed time alongside the output.

This is an evaluation plan, not a report of a test I have run. Before expanding to a folder of recordings, check that the transcript reaches the end of the sample. An apparently fluent partial transcript is still incomplete.

## Choose the transcript contract explicitly

The model card documents these controls:

| Control | Documented behavior | Evaluation question |
| --- | --- | --- |
| `language` | Can force the language; otherwise output can include a language marker | Does your parser expect only spoken text? |
| `digits` | Defaults to enabled for numeric formatting | Should the reference contain numerals or written-out numbers? |
| `filler` | Defaults to disabled | Must hesitation words be retained? |
| `verbatim` | Defaults to enabled; disabling it can collapse repeated words | Are repetitions meaningful in this application? |

The card shows controls through processor templates or vLLM chat-template arguments. Keep those settings with the transcript rather than assuming every integration shares your defaults.

For example, a searchable meeting summary and an interview quotation have different requirements. Removing hesitation may help a reader skim a meeting, but a quotation should not silently become a polished version of what someone said. Preserve the original audio and label any cleaned transcript accordingly.

I would also keep raw model output separate from application-ready text. That makes it possible to diagnose whether an unexpected token came from recognition, language metadata or your own post-processing.

## Separate throughput from waiting time

The launch reports throughput above 7,000 times real time when fully batching large benchmark datasets. That is a measure of aggregate audio processing under the reported conditions, not a promise that an individual uploaded recording will return almost instantly.

Its reported AppTek English-accent evaluation gives Rumble-2-Flash a 7.86% word error rate, compared with 7.78% for the larger Nemotron model in that table. Those are vendor-reported benchmark results, not a guarantee of equivalence on your recordings.

Measure the experience you need. For an interactive product, track the time from submitting one clip to receiving its complete transcript. For a backlog-processing job, track total audio duration processed per wall-clock hour at a stated batch size. Include loading and queue time where they affect the user.

Those two tests can favor different configurations. A server that clears a large overnight backlog efficiently may still be a poor fit for a user waiting on one short recording.

## Build a reference set around expensive mistakes

Choose recordings with the names, numbers, accents and background conditions your application encounters. Include silence and incomplete utterances rather than testing only clean narration. Keep a held-out set that you do not repeatedly use to tune settings.

Apply the same normalization rules to reference and output when calculating an aggregate score. Separately inspect mistakes that matter operationally, such as an incorrect product identifier or amount. A good average can hide the errors your reviewers spend the most time fixing.

My preference would be to make formatting reproducible first, then measure accuracy and latency. That gives a small open model a fair trial without turning a batch-throughput headline into a product promise.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI teams turn model releases into practical developer guides and credible product stories.

[Book a 30-minute call](https://cal.com/paras-madan/30min).
