# Cartesia H-Net Research: Learned Tokenization and the 10x Training Claim

Cartesia's new byte-level model results suggest that how a model divides text deserves more attention than it usually gets.

By Paras Madan | 2026-10-29

Cartesia published new H-Net research on October 28, showing how models that learn to divide raw bytes can use training compute more efficiently. Its headline comparison reports roughly tenfold savings against SmolLM3 on selected evaluations. The improved architecture remains a planned release, rather than a new downloadable model or API.

## How H-Net replaces a fixed tokenizer

A normal language model first receives text divided into tokens. These can be words, pieces of words, punctuation or other recurring sequences. A fixed tokenizer determines those boundaries before the main model processes the text.

The [original H-Net paper](https://arxiv.org/abs/2507.07955), published in July 2025, proposed learning those boundaries jointly with the model. Its dynamic chunking mechanism uses both the content and its context. Multiple stages can build progressively higher-level representations from the underlying bytes.

That distinction is useful when thinking about code. A function name might contain a familiar English word, but its meaning also depends on the surrounding program. A system that learns its own grouping has another way to represent that structure, instead of inheriting every boundary from a separate preprocessing system.

## What the new training results show

In [Cartesia's first research post](https://www.cartesia.ai/blog/from-tokens-to-bytes-part-1), a tightly controlled one-stage comparison reports a 1.28x compute advantage on validation loss. Larger gains appear on other distributions. These are training-efficiency measurements, not claims that deployed inference is 1.28 times faster.

The larger two-stage experiment roughly matches SmolLM3's final average score on six tasks after around one trillion token-equivalents of training, compared with about eleven trillion tokens for SmolLM3. Compute per byte is matched within 2%, but parameter counts differ: 3.67 billion for H-Net and 3.08 billion for the baseline.

**The tenfold claim belongs to that specific comparison.** Cartesia reproduces the baseline's first-stage data mixture on a best-effort basis, while the full SmolLM3 run uses multiple training phases. The authors also acknowledge that matching data and compute across domains is difficult when the architectures compress them differently.

## Why the second research post matters

Cartesia's [second post](https://www.cartesia.ai/blog/from-tokens-to-bytes-part-2) examines the tradeoff behind those results. It compares byte-level models, several tokenization approaches and hierarchical networks, separating efficiency during pretraining from performance on other data.

One focus is pretokenization: hand-written rules that divide text before the tokenizer builds its vocabulary. These rules can encode useful assumptions about spaces, numbers and language. Removing them is therefore a change to the information structure a model receives, not simply deleting an unnecessary step.

Cartesia argues that access to finer-grained data helps generalization, while compressed inputs make training more efficient. Its experiments position end-to-end hierarchical learning as a way to improve that tradeoff. This remains the researchers' interpretation of their experiments, rather than independent confirmation across production models.

## What developers should watch next

There is no reason to change an application's tokenizer based on these results alone. H-Net learns its segmentation as part of training, so this is an architectural choice for model builders, not a setting that makes an existing model cheaper overnight.

For a team evaluating a future release, the practical questions would be broader than pretraining scores: memory use, serving throughput, long-input behavior and reliability on the team's own data. Training efficiency and application economics are connected, but the connection has to be measured.

I think the useful implication is about where model improvement can come from. More compute is one option. Learning a better representation of the same input is another. A convincing next step would show that the advantage survives independent reproduction and the constraints of serving a useful model.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help technical teams turn research and product updates into clear stories that developers can understand and share.

[Book a 30-minute call](https://cal.com/paras-madan/30min).