OpenAI published its internal perplexity scores on November 22, 2026, disclosing losses below 0.8 on its evaluation dataset, and proposed the metric as a shared AI safety standard S¹. The company concedes the disclosure hands competitors a clearer picture of where OpenAI stands, but says the urgency of shared safety measures outweighs that cost S¹.
My read: The number I would watch is the gap between GPT-4's 0.91 and the sub-0.8 losses OpenAI says it has now reached. That band is where the company claims "transformative capabilities" in cybersecurity and mathematics emerged. But nobody outside OpenAI has verified these measurements, and the post is an argument for a standard, not the standard itself. No other developer has signed on.
What perplexity actually measures
Perplexity, in this context, is not the search company of the same name. It is an information-theoretic score: how well a model predicts the next token (the next chunk of text) on average, given everything before it S¹. Lower is better. A model with a perplexity of 1 would be perfectly certain about every next word; real models sit well above that. OpenAI says it has tracked perplexity on a held-out dataset for years as the primary measure of progress on what it calls the "intelligence substrate" S¹.
The metric is equivalent to cross-entropy loss, a standard training objective, expressed on a logarithmic scale S¹.
The numbers behind four generations
OpenAI's blog post maps each generation to a perplexity milestone on its current main evaluation dataset S¹. GPT-2 set a new state-of-the-art on older datasets and was, in OpenAI's words, the first model to consistently produce coherent paragraphs S¹. GPT-3 added in-context learning, the ability to pick up a task from a few examples in the prompt S¹. GPT-3.5, at a log-perplexity of approximately 0.99, was fluent enough in conversation to become the original ChatGPT S¹P⁷. GPT-4, at approximately 0.91, used chain-of-thought reasoning well enough to motivate the reasoning models that followed S¹.

Now OpenAI says it has reached losses below 0.8 on the same dataset, and that this is where it observed "transformative capabilities" in cybersecurity and mathematics S¹. The post does not name a specific model at that level, nor does it publish error bars or methodology details that would let an outside party reproduce the measurement.
Why OpenAI is giving away the scoreboard
The blog post frames perplexity as a candidate for a shared industry metric, one that would let developers compare models on a common scale and coordinate safety thresholds S¹. OpenAI calls this a step toward "a shared scientific basis for future pacing efforts" S¹.
The admission that publishing helps competitors is unusual. Most labs treat training-loss curves as proprietary. But the post stops short of proposing a binding standard or naming partners. No other developer has adopted the metric for safety comparisons, and OpenAI has not disclosed the dataset, compute budget, or training methodology behind the numbers S¹.
The post drew discussion on Hacker News, gathering between 80 and 96 points and 36 to 43 comments across two snapshots taken roughly an hour apart on November 22 S³S⁴.
A team evaluating whether to trust OpenAI's safety claims, say a policy group advising regulators, would need the evaluation dataset itself to verify the scores. Without it, the shared standard OpenAI proposes has no shared way to check it.
Sources: S1 — Perplexity and Intelligence · S2 — Perplexity and Intelligence - OpenAI · S3 — Perplexity and Intelligence · S4 — Perplexity and Intelligence · P5 — Perplexity trusts GPT-6 Astra with end-to-end systems | OpenAI · P6 — openai (OpenAI) · P7 — Introducing ChatGPT | OpenAI · P8 — gaojing8500/AIGCOpenSource
Written from 8 sourced items, 5 of them primary.