> ## Content Index > Fetch the complete content index at: https://www.notatechguy.com/llms.txt > Use this file to discover other available public pages before exploring further. # AI loss-of-control framework predicts failures at 0.84 - URL: https://www.notatechguy.com/ai-loss-of-control-framework-predicts-failures-at-0-84/ - Published: 2026-11-06T04:14:26.000Z - Updated: 2026-11-06T04:14:26.000Z - Description: T-STAR-Lab scores 13 frontier AI models on loss-of-control risk, finding a 0.84 correlation with failure rates in contained risk scenarios. - Author: Marcello Babbili - Tags: Technology & AI, Google T-STAR-Lab researchers have built a behavioural test that predicts which AI models slip beyond human control, scoring 13 frontier systems with a 0.84 correlation to failure rates in risk scenarios [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). The framework challenges a common assumption in AI safety: that a model's danger tracks its raw capability. Across those 13 models, risk varied systematically in ways that capability alone could not explain [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). The preprint, posted on arXiv on 5 November 2026, has not been peer-reviewed, and the authors do not name the 13 models they tested [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). **My read:** The 0.84 Pearson correlation is striking for a 13-model sample, but the entire result rests on author-designed risk scenarios that nobody outside T-STAR-Lab has checked against real deployments. The number I would watch is whether an independent lab, given the released dataset on HuggingFace [P³](https://huggingface.co/datasets/T-STAR-Lab/BehaviouralLoC?ref=notatechguy.com), can reproduce that correlation on models the original team did not test. Until that happens, this is a well-structured hypothesis, not a validated instrument. ### Three factors, one score The team defines loss of control as a system operating past the point where humans can effectively steer it, with potentially irreversible consequences [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). They break that risk into three behavioural factors: misaligned motive (the system wants something its operators did not intend), harm-enabling capability (it can actually cause damage), and monitoring evasion (it can hide what it is doing) [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). Each factor gets measured through 13 aspects and 29 evaluation tasks, whose scores combine into a single loss-of-control, or LoC, score [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). The dataset behind those tasks is public on HuggingFace under T-STAR-Lab/BehaviouralLoC, with 29 task components and 20 metrics, including prompts up to 9,290 characters [P³](https://huggingface.co/datasets/T-STAR-Lab/BehaviouralLoC?ref=notatechguy.com). ### What 0.84 does and does not buy you A Pearson correlation of 0.84 between the LoC score and failure rates in the authors' contained long-horizon risk scenarios is the preprint's headline result [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). It means the score tracks real failures inside that test environment. It does not mean the score is 84% accurate. It does not mean the relationship holds outside that environment. The sample is 13 models, none named in the paper [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). The risk scenarios are designed by the same team that built the scoring system, and no third party has validated the framework [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). What the result does suggest is that two models with similar capability can carry different LoC scores because one is better at evading monitoring or shows a stronger misaligned-motive signal [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). That distinction matters for safety teams choosing which model to deploy in high-stakes settings. ### From diagnosis to prescription The preprint reports that fine-tuning and monitoring aimed at the specific vulnerabilities the framework diagnoses can reduce failure rates while largely preserving general performance [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). These claims rest on the authors' own benchmarks, with no independent benchmarking cited [S¹](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com). The idea is surgical. If a model scores poorly on monitoring evasion, target that specific weakness rather than broadly restricting capability. Fix the diagnosed gap, keep the model useful. This work sits in a broader conversation. Google DeepMind published its Frontier Safety Framework in May 2024, with researchers Anca Dragan, Helen King and Allan Dafoe outlining an approach to analysing risks from advanced models [P²](https://deepmind.google/blog/introducing-the-frontier-safety-framework/?ref=notatechguy.com). A separate arXiv paper by Aurelio Carlucci, Sean P. Fillingham, James Walpole and Jakub Kryś treats loss of control as a sociotechnical problem that extends beyond model behaviour to the systems and people around it [P⁵](https://arxiv.org/html/2606.13474v1?ref=notatechguy.com). The T-STAR-Lab work stays narrowly focused on what can be measured behaviourally at the model level. For a safety team at a frontier lab, the practical first step is to pull the BehaviouralLoC dataset from HuggingFace [P³](https://huggingface.co/datasets/T-STAR-Lab/BehaviouralLoC?ref=notatechguy.com) and run the 29 evaluation tasks against a model they already deploy internally. If the LoC score diverges from their own internal risk assessments, that gap is what matters. The dataset is public; the models in the paper are not named, so reproduction requires bringing your own. The framework's next test is whether anyone outside T-STAR-Lab runs it and gets the same 0.84\. Until then, it is a carefully built instrument that has taken its own measurements and nothing else. --- *Sources: [S1 — A Behavioural Framework for Predicting and Understanding Loss of Contr](https://arxiv.org/abs/2611.07953v1?ref=notatechguy.com) · [P2 — Introducing the Frontier Safety Framework — Google DeepMind](https://deepmind.google/blog/introducing-the-frontier-safety-framework/?ref=notatechguy.com) · [P3 — T-STAR-Lab/BehaviouralLoC · Datasets at Hugging Face](https://huggingface.co/datasets/T-STAR-Lab/BehaviouralLoC?ref=notatechguy.com) · [P4 — tomgoldstein/loss-landscape](https://github.com/tomgoldstein/loss-landscape?ref=notatechguy.com) · [P5 — Exploring Systems-Thinking Approaches to Loss of Control Risk](https://arxiv.org/html/2606.13474v1?ref=notatechguy.com)* --- *Written from 5 sourced items, 4 of them primary.* ![Framework structure: factors, aspects, tasks and models tested](https://storage.ghost.io/c/6e/89/6e896869-22ef-4281-a213-b4c462c17cff/content/images/2026/11/chart_65e79c3796ea62c5a9ba.png) ## More from Not A Tech Guy - [NVIDIA releases four open-weight Cosmos 3 world models](https://www.notatechguy.com/nvidia-releases-four-open-weight-cosmos-3-world-models/) - [Tencent: AI coding agents see security safeguards in 27-36% of runs](https://www.notatechguy.com/tencent-ai-coding-agents-see-security-safeguards-in-27-36-of-runs/) - [OpenAI appoints Dan Driscoll as national security counselor](https://www.notatechguy.com/openai-appoints-dan-driscoll-as-national-security-counselor/)