Harmful prompts in Twi, Hausa, Amharic and Swahili retain less than 10% of the English refusal signal that AI models produce when they detect danger, according to a new arXiv preprint published on 11 August S¹. The finding comes from probing the internal states of language models, including their hidden representations, and it points to a gap that output-only testing would miss entirely. The question is whether the billions spent on English safety alignment are reaching anyone who does not speak it.

My read: This is the sharpest demonstration I have seen of this problem. This paper probes the internal representations and finds the models encode the harmful concepts just fine. They simply do not act on them. I am cautious about the "less than 10%" figure as a universal metric because it depends on the authors' specific geometric framework, but the direction matches what a separate preprint found in May: low-resource safety failures are action failures, not representation failures P².

How the study catches what output testing misses

Safety alignment in large language models is largely built in English, on the assumption that the safeguards transfer across languages S¹. The researchers tested that assumption in Twi, Hausa, Amharic and Swahili using a new dataset called LoDNA, which pairs literal translations of harmful prompts with culturally localized versions of the same prompts S¹.

Here is the subtle part. The literal and localized prompts are semantically aligned, with cosine similarity between 0.95 and 0.996 S¹. In plain terms, they mean almost the same thing. Yet they drift apart as they pass through the model's layers S¹. The model processes them differently internally even though a human would read them as equivalent.

To see what happens inside, the researchers built a latent geometric framework that probes hidden-state refusal representations, the internal signals that tell a model to decline a harmful request S¹. This is a step beyond generation-based evaluation, which only checks what the model outputs S¹. The internal view reveals what output testing cannot.

The models understand the harm. They just do not act on it

The core finding: harmful prompts in these four languages retain less than 10% of the English refusal signal across most language-model pairs studied S¹. The models encode the concepts but do not route them to safety mechanisms S¹. The researchers conclude that current multilingual safety alignment is superficial S¹.

This aligns with a separate preprint from May 2026, which framed low-resource safety failures as action failures rather than representation failures P². It also echoes findings from the Minionese benchmark, which reported that safety alignment in LLMs remains brittle across languages and that prompts reliably refused in English can elicit harmful responses in other languages P⁴. The LinguaSafe benchmark, another multilingual safety evaluation, was built to probe exactly this kind of gap P³.

The researchers go further, arguing their findings provide strong evidence against the assumption of a universal, language-agnostic harm manifold, at least within the four languages they studied S¹. If harm is not encoded the same way across languages, then English safety training does not automatically protect speakers of other languages.

What to do about it

Consider a telehealth startup in Accra that deploys an LLM chatbot to triage symptoms in Twi. The model was trained with English safety alignment, so it refuses to provide dangerous medical advice when asked in English. A user asks the same question in Twi, perhaps phrased with local idioms about traditional remedies. The model understands the request. The semantic content is nearly identical to its English counterpart. But the internal refusal signal is largely absent, and the model may comply.

For any team deploying a multilingual model, the practical step this week is to test with culturally localized harmful prompts, including literal translations. If your safety evaluation only uses machine-translated English prompts, you are testing translation, not safety. The gap the paper identifies is between meaning the model recognizes and danger the model acts on. A localized prompt catches that gap. A literal one may not.

What we don't know yet

The study covers four African languages. Whether the findings extend to other low-resource languages, or to high-resource non-English languages, is an open question the paper does not answer S¹. The "less than 10%" figure is specific to the authors' latent geometric framework and may not map directly to standard benchmark scores or real-world generation behavior. This is a preprint that has not been peer-reviewed S¹, and the paper itself carries a content warning for offensive example data S¹.

Microsoft's BYOL toolkit, a GitHub project for bringing low-resource languages into LLMs, lists Chichewa, Inuktitut and Nyanja among its target languages P⁵, suggesting the ecosystem for low-resource language support is still nascent. The gap between representation and action may be wider in languages with even less training data than the four studied here.

The next signal: the first independent reproduction or challenge of the sub-10% finding, which we expect to surface as a follow-up preprint or benchmark extension on arXiv. We will check it against this paper's claims when it arrives. If you want to follow that thread with us, subscribe and we will bring you the result.

Sources: S1 — The Illusion of Cross-Lingual Safety in Low-Resource Languages · P2 — Low-Resource Safety Failures Are Action Failures, Not Representation F · P3 — LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large La · P4 — Minionese: Comprehensive Benchmark and Mechanistic Study of Multilingu · P5 — microsoft/byol

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.