A July 17 arXiv preprint from researchers at the University of Manchester and Warwick argues that the way AI systems measure their own uncertainty has been built on a shaky assumption, and offers a single mathematical framework that pulls dozens of competing measures under one roof [S1][P2]. If the idea survives review, it could simplify how every AI team decides when to trust a model and when to hand off to a human. The catch is that the paper is purely theoretical, with no experiments to back it up yet [S1].

The problem with measuring uncertainty today

When an AI model makes a prediction, it is wrong in two different ways. Epistemic uncertainty is the gap between what the model knows and what it could know with more data or a better architecture. Aleatoric uncertainty is the noise in the world itself, the irreducible randomness that no amount of training data will fix. Telling the two apart matters enormously. A medical model that is uncertain because it has never seen enough cases of a rare disease needs more data. One that is uncertain because the symptoms genuinely overlap with three conditions needs a different decision rule, not more scans.

The trouble is that the field of uncertainty quantification, or UQ, has spent years proposing separate mathematical measures for each type of uncertainty, each with its own axioms and justifications [S1]. There is no shared foundation. A team picking one measure over another is often choosing between frameworks that do not even speak the same language.

The reframing: uncertainty as consequence, not primitive

The preprint's central move is a reframing. The authors, Raghad Alamri and Michele Caprio, argue that uncertainty measures are not fundamental objects that need their own axioms and arguments [S1][P2]. Instead, they are consequences of higher-level modelling decisions. Pick your model and your loss function, and the right uncertainty measures fall out of the maths automatically.

The mechanism is a decomposition of what the authors call subjective risk, built on a strictly proper loss. A strictly proper loss is one that is minimised only when the model reports its true beliefs, not when it games the score. By decomposing subjective risk, the authors show that epistemic and aleatoric uncertainty terms emerge naturally, without anyone having to define them from scratch [S1].

One concrete example makes this tangible. When the authors apply their decomposition to reverse cross-entropy, a common loss function, the framework recovers the classic information-theoretic uncertainty terms that researchers have used for years [S1]. The same approach also recovers numerous other measures scattered across the UQ literature, giving them a shared theoretical home for the first time [S1].

The authors go further, introducing subjective-risk analogues of excess risk, approximation error, and estimation error, three concepts from classical learning theory, and mapping their connections to uncertainty [S1]. They call the paper a first step toward a full learning-theoretic framework for UQ [S1].

What it means

For a reader with no background in uncertainty maths, the takeaway is this: the paper says we have been treating the symptoms. Every time someone proposed a new uncertainty measure, they built it from the ground up with bespoke axioms. This paper says you do not need to. Choose your model and your loss function, and the uncertainty measures are already implied by that choice. You just have to extract them.

If this holds up, it does not replace existing measures. It explains them. A team using an information-theoretic uncertainty score today is not wrong; they are using a special case of a more general framework without realising it. The practical promise is that teams could stop arguing about which uncertainty measure to use and instead ask which loss function fits their problem, then read off the corresponding uncertainty terms.

What it means for business

A two-person AI consultancy building a custom classifier for a legal client currently faces a bewildering menu of uncertainty measures, each with its own implementation and its own defenders. This paper suggests a simpler workflow: define the modelling scenario and the loss function, and the epistemic and aleatoric terms are induced by the decomposition [S1]. No separate uncertainty library to evaluate, no competing axioms to reconcile.

For a suburban medical imaging startup, the distinction between epistemic and aleatoric uncertainty directly drives product design. High epistemic uncertainty on a scan means the model needs more training data before it can be trusted on that case type. High aleatoric uncertainty means the case is genuinely ambiguous and the system should route it to a human reviewer regardless of data volume. A framework that derives both terms from the same maths, rather than bolting them on separately, could make that routing logic cleaner and easier to audit.

That said, no team should rip out their current uncertainty pipeline based on a preprint. The authors describe no experiments, no benchmarks, no real-world deployment [S1]. The framework is a theoretical scaffold, not a tested product.

What we don't know yet

The paper has not been peer-reviewed, and arXiv preprints can be revised, superseded, or withdrawn without notice [S1]. The authors' claims of novelty and unification are self-assessed, with no external corroboration in the source material.

There are no empirical results in the provided excerpt. We do not know whether the framework produces better-calibrated uncertainty estimates in practice, whether it is computationally tractable for large models, or whether it handles the messy realities of deep learning, where strictly proper losses are common but the assumptions behind subjective risk may not always hold.

The paper also does not address how its framework interacts with modern large language models, where uncertainty is often estimated through sampling methods rather than loss decomposition. Whether the approach extends to generative models is an open question.

The next concrete signal to watch is whether the paper appears at a peer-reviewed venue such as NeurIPS or ICML, and whether the authors or independent groups release code and benchmark results. A GitHub repository for a related but separate project, LUQ, which quantifies uncertainty in language models, was last updated in June 2025 [P4], suggesting active community interest in practical UQ tooling. Until this framework meets real data, it is a promising idea, not a proven tool.

If you found this useful, subscribe for plain-English decode of every AI research paper that actually changes what teams should build next.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.