HealthBench Professional Leaderboard: Why Answer Length Changes the Rankings

Artificial Analysis added HealthBench and HealthBench Professional leaderboards on October 19, 2026. The new independent evaluations compare models on health conversations, but their most useful finding is methodological: the model with the highest raw Professional score falls to sixth after an adjustment for answer length.

These are new evaluations of existing OpenAI benchmarks, rather than newly released medical models. For developers reading the results, the important question is what the score rewards before deciding what the ranking means.

HealthBench Professional results

In the October 19 results, Claude Opus 5.5 at max effort with default fallback earns the highest raw HealthBench Professional score, 71.4%. Its length-adjusted score is 59.9%.

GPT-6 Astra at max effort leads the adjusted ranking. The selected results show how much the choice of metric matters:

Model configuration	Length-adjusted Professional score
GPT-6 Astra, max	66.0%
GPT-6.1 Sol, max	65.1%
Claude Sonnet 5.5, max with default fallback	63.0%
Claude Opus 5.5, max with default fallback	59.9%

These figures were checked on October 20, 2026. The table shows selected configurations, not the complete ranking. Opus averages roughly 5,900 characters per response, which helps explain the size of its adjustment. Raw and adjusted scores answer different questions, even though they come from the same responses.

How the length adjustment works

The Professional leaderboard applies no length adjustment at 2,000 characters. Each additional 500 characters subtracts 1.47 percentage points; shorter responses receive the corresponding addition. Hidden reasoning does not count toward this length.

For a hypothetical answer with a raw score of 60% and 3,000 characters, the adjustment subtracts 2.94 points, producing 57.06% before aggregation. This is an illustration of the arithmetic, not a measured model result.

That makes the tradeoff visible. Adding another 1,000 characters has to earn more than 2.94 additional raw points just to improve the adjusted result. A longer response can cover more rubric items, but the metric also accounts for how much text it takes to do that.

The adjustment is an evaluation rule, not a recommendation about the appropriate length of a clinical answer.

What is actually being evaluated

Artificial Analysis's testing methodology feeds each model a conversation and asks for the next reply, without tools or a system prompt. A judge checks the reply against physician-written criteria. Satisfied positive criteria add points; satisfied negative criteria subtract them.

HealthBench contains 5,000 conversations and uses GPT-4.1 as grader. Professional contains 525 clinician conversations, uses GPT-5.4 at low reasoning effort and averages four attempts per task. The general HealthBench evaluation runs once per task.

The two also use different length penalties. HealthBench subtracts 2.99 points per 500 characters above 2,000, compared with Professional's 1.47. Readers should therefore avoid treating scores from the two datasets as interchangeable measurements.

Why the result cannot establish clinical reliability

The original Professional paper, submitted on April 30, 2026, describes deliberate selection for difficulty. Difficult examples were enriched roughly 3.5 times relative to the candidate pool, and about a third involve adversarial testing by physicians.

That makes it a useful stress test, but an aggregate score cannot be read as the percentage of ordinary clinical situations a model handles safely. The experiment measures responses to selected conversations under a particular grading procedure. It does not measure patient outcomes.

There is another boundary: a model answering without tools is a different system from a product that retrieves references, uses an application-specific prompt or has human review. Changes to those components need their own evaluation rather than inheriting a leaderboard result.

Read both scores before the ranking

For an evaluation team, the useful comparison is the raw score, the adjusted score and the response length together. If an apparent lead disappears after adjustment, inspect whether the extra text supplies necessary information or mostly creates more reading.

The new leaderboard makes that tension easier to see. A single headline number hides the choice between rewarding coverage and rewarding efficient communication. Understanding that choice is more useful than treating whichever model finishes first as the answer to every workflow.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders explain complex AI products and evaluations through clear technical stories that developers and founders read and share.

Book a 30-minute call.