A July 15 arXiv preprint from researchers at HPI / University of Potsdam and the African Institute for Mathematical Sciences trains language models on text with every named entity anonymised, and the resulting models recall fewer facts from memory but answer questions from provided context up to 25% better than standard models [S1][P2]. The paper calls them Knowledgeless Language Models, or KLLMs. The counterintuitive payoff: a model that cannot remember who won the 2018 World Cup may be far more useful to a business that needs it to read a contract and answer questions about it. Whether that trade-off holds up outside benchmark conditions is the question the preprint does not yet answer.

The dumbest smart thing a model can do

Today's large language models have a well-documented habit: they memorise facts during pretraining, then blurt those memorised facts out even when the evidence in front of them says otherwise. Ask a model to summarise a document and it may slip in a detail it learned years ago that contradicts the document. Researchers call this parametric recall, meaning knowledge baked into the model's weights rather than drawn from context [S1].

The KLLM approach attacks this at the root. During pretraining, the researchers replace named entities in the training corpus with anonymised tokens. The model still learns grammar, reasoning, and how language works. It just never learns that "Paris" is the capital of France, because "Paris" became a placeholder before the model ever saw it [S1].

The result is a model that substantially reduces closed-book factual recall, the kind where you ask a question with no supporting documents [S1]. In plain terms: it gets worse at trivia. But on tasks where the answer is in the provided context, it often performs better than standard models [S1].

Where the numbers land

The preprint reports results across multiple model scales. KLLMs consistently outperform matched baselines on three categories of benchmark: contextual question answering, fact verification, and hallucination detection [S1].

The most striking figure comes from retrieval-grounded settings with imperfect evidence, meaning the retrieved documents contain noise, contradictions, or gaps. There, KLLMs achieve up to 20 to 25% relative gains over standard language models [S1]. That number is not a universal improvement. It describes a specific scenario: the model is given retrieved evidence that is messy, and it needs to reason over that evidence rather than fall back on what it memorised.

The models also show better calibration, measured by three standard metrics: ECE (expected calibration error, how well a model's confidence matches its accuracy), Brier score (a combined measure of accuracy and confidence), and AUROC (how well the model distinguishes right answers from wrong ones) [S1]. They also demonstrate more reliable abstention, meaning they are more willing to say "I don't know" when they should [S1].

The researchers frame this as a shift in epistemic behaviour. Suppressing entity-linked supervision during pretraining causes KLLMs to rely less on parametric knowledge and more on external evidence [S1]. They suggest this pretraining-time control can complement retrieval-augmented generation and tool-based systems [S1].

A related effort, OpenBMB's ParamMute project on GitHub, takes a different angle on the same problem: suppressing specific feed-forward network components that store knowledge, to make retrieval-augmented generation more faithful [P5]. KLLMs target the same disease at an earlier stage, during pretraining rather than after.

The paper is also under review at ICLR 2026, according to an OpenReview listing [P4].

What it means

The core idea is simple enough to explain to anyone who has watched a confident person give a wrong answer. Standard language models are like a student who memorised the textbook but refuses to check it during an open-book exam. They answer from memory even when the answer sheet is in front of them, and sometimes the memory is wrong.

KLLMs are the student who never memorised the textbook but learned to read carefully. When you hand them a document, they actually read it. They cannot answer from memory because they do not have one, not for named facts anyway. So they lean on the evidence you provide.

This matters because the fastest-growing use of language models in business is not trivia. It is document work: reading contracts, summarising research, answering questions from internal knowledge bases. For those tasks, a model that hallucinates less and reads more carefully is more valuable than one that remembers the capital of every country.

The calibration and abstention findings matter too. A model that knows when it does not know is safer to deploy. If it says "I'm not sure" instead of inventing an answer, a human can step in. Standard models often do the opposite: they generate fluent, confident nonsense.

What it means for business

Consider a two-person law firm that uses an AI tool to scan contracts for unusual clauses. Today, the model might flag a clause as "standard" because it has seen similar language thousands times in training, even though this specific contract has a subtle variation that changes the meaning. A KLLM-style model, which cannot lean on memorised patterns of what clauses usually say, would be more likely to report what the clause in front of it actually says.

Or take a suburban real estate agency that feeds property reports into a model and asks for summaries. A standard model might insert a detail about a suburb it learned during pretraining, a median price, a school zone, that is outdated or wrong for this specific listing. A model trained without entity knowledge has nothing to insert. It works with what you give it.

The 20 to 25% gain in imperfect-evidence settings is the number that should get attention from anyone building retrieval-augmented systems. Real-world retrieval is almost always imperfect. Documents are incomplete, search results are noisy, and context windows are limited. If KLLMs hold up, they could reduce the engineering effort spent on post-processing model outputs to catch hallucinations.

The abstention improvement has a direct cost implication. When a model says "I don't know" reliably, you can route that query to a human. When it hallucinates confidently, the error propagates downstream, sometimes for weeks before anyone notices.

What we don't know yet

This is a single preprint, not yet peer-reviewed, though it is under review at ICLR 2026 [S1][P4]. Several questions remain open.

The abstract does not disclose specific model scales, training corpus size, or exact benchmark names [S1]. Without knowing whether these results come from 1-billion-parameter models or 70-billion-parameter models, it is hard to judge how they transfer to production-scale systems.

The paper does not discuss potential negative trade-offs on capabilities that are not entity-linked. If a model never sees named entities during pretraining, does its reasoning suffer? Does it struggle with tasks that require entity knowledge indirectly, such as coreference resolution (working out that "he" refers to "John")? The preprint does not say.

The 20 to 25% figure applies only to imperfect-evidence retrieval settings [S1]. It is not a general improvement number, and treating it as one would be misleading.

The anonymisation process itself has practical questions. How is it done? Does it require a separate named-entity recognition system, and how accurate must that system be? If entities leak through, the approach degrades.

Finally, no one has shown these results in a commercial deployment. The gap between a benchmark and a production system is where most AI research quietly fails.

The next concrete signal to watch: the ICLR 2026 review decision, which will either surface reviewer concerns about the missing details or validate the approach. Until then, this is a promising idea with a clean mechanism and an honest set of caveats.

If this kind of plain-English decode is what you want from AI coverage, subscribe to keep reading.

Sources

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.