> ## Content Index
> Fetch the complete content index at: https://www.notatechguy.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# AI agents learn your standards, up to 20.9% better
- URL: https://www.notatechguy.com/ai-agents-learn-your-standards-up-to-20-9-better/
- Published: 2026-09-05T18:00:04.000Z
- Updated: 2026-09-05T18:00:04.000Z
- Description: New preprint: AI agents learn individual standards through interaction, lifting task success up to 20.9% for professionals needing personal quality.
- Author: Marcello Babbili
- Tags: Technology & AI, AI Agents, OpenAI

An arXiv preprint published September 3 shows AI agents can learn individual user standards by mining their own interaction history, lifting solo task success by 4.5% to 20.9% within tens of tasks [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). The method, called TAHI (test-time adaptation through human-agent interaction), captures something every AI tool currently discards: the trail of corrections, refinements, and implicit preferences a person leaves across sessions. Whether those gains survive beyond 30 people and two domains is the question the paper leaves dangling.

**My read:** This is the first method I've seen that treats cross-session interaction data as a training signal for personalisation, rather than just context for the current chat. I don't buy the "efficient" label yet, because the paper never quantifies computational cost. And the 8.8% generalisation figure comes from the same 30-person cohort, not a separate holdout group, so I'd call it suggestive rather than proven. What I find genuinely interesting is the evolving rubric module. The idea that users have evaluation criteria they apply consistently but cannot articulate upfront rings true for anyone who has tried to explain "good writing" to a chatbot.

## The problem: one model, thirty tastes

The paper starts from a complaint every professional has voiced. AI agents trained on population-scale data produce work that rarely meets the personal bar professionals need to stake their reputation on [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). A copywriter wants punchy and concrete. A technical illustrator wants precise and clean. The same model serves both, and satisfies neither fully.

The authors argue this gap exists because users have criteria they cannot fully specify upfront but apply repeatedly across tasks [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). You know it when you see it. You just cannot write it in a prompt.

## How TAHI works

TAHI integrates cross-session human-agent interaction signals into two places: the agent's context (what it sees) and its weights (what it prioritises) [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). The core mechanism is an evolving rubric module that crystallises each user's training and evaluation criteria over time [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com).

Think of it as a feedback loop. A user works with the agent on a task. They correct the output. The rubric module captures what those corrections reveal about the user's standards. Next session, the agent carries those standards forward, both in its prompt context and in adjusted model weights. Over tens of tasks, the agent converges on what that specific person actually wants.

The rubric itself is where the method gets interesting. The paper reports that these evolving rubrics catch 16.0% to 22.3% more failures than rubrics from language models or humans alone [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). The combination of human judgement and machine pattern-finding surfaces quality criteria that neither party identifies on its own.

## The numbers, in context

The researchers adapted agents to 30 individuals across writing and visual creation domains, running 600 tasks total [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). The headline result: solo task success improved by 4.5% to 20.9% within tens of tasks [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). Personalised agents also produced improvements of up to 8.8% that generalised across users [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). Some of what the system learned about one person's standards transferred to others.

![TAHI reported improvements by metric](https://storage.ghost.io/c/6e/89/6e896869-22ef-4281-a213-b4c462c17cff/content/images/2026/09/chart_cd8f1882a74a3bf05fc9.png)

Those are meaningful gains if you are a professional whose daily work involves dozens of AI-assisted tasks. A 20% improvement in task success rate is the difference between an agent you tolerate and one you actually deploy. But the ranges are wide, the sample is small, and "tens of tasks" is vague enough to hide variability in how quickly adaptation happens.

## What to do about it

Consider a copywriting studio that uses AI to draft client work. The senior writer has firm rules: no clichés, active voice, a concrete number in every paragraph, never open with a question. They correct the AI's output the same way every time, across every brief. Today, those corrections vanish after each session. TAHI would capture them, build a rubric from them, and apply them automatically to the next draft. The writer stops repeating themselves. The agent starts producing work that matches the studio's house style without being told each time.

That is the promise. The practical version, today, is simpler. If you work with AI agents on repeated tasks, start logging your corrections. Note what you change and why. Even without TAHI, that log is the raw material for better system prompts, few-shot examples, or custom evaluation criteria. The paper's insight, that users have implicit standards they cannot articulate but apply consistently, is actionable right now with a text file and ten minutes of reflection.

## What we don't know yet

The paper is a preprint. None of its findings have been validated by peer review [S¹](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com). The 30-person sample is small, the two domains are narrow, and all performance metrics are self-reported by the authors without independent replication. The word "efficient" in the title is not backed by any quantified computational cost. The generalisation claim of 8.8% derives from the same cohort, not a separate holdout group. And the method has not been tested outside writing and visual creation, so we cannot say whether it transfers to coding, data analysis, or other domains where professionals use AI agents.

The next signal: whether the authors release code and data for independent replication, or whether a peer-reviewed version appears at a major machine learning conference. We will check both claims against whatever comes next. If you want to follow along, subscribe for the update.

---

*Sources: [S1 — Efficient Test-Time Adaptation through Human-AI Interaction](https://arxiv.org/abs/2609.04141v1?ref=notatechguy.com) · [P2 — \[2609.04141\] Efficient Test-Time Adaptation through Human-AI Interacti](https://arxiv.org/abs/2609.04141?ref=notatechguy.com) · [P3 — kdiAAA/TDA](https://github.com/kdiAAA/TDA?ref=notatechguy.com) · [P4 — Enabling Self-Improving Agents to Learn at Test Time With Human-In-The](https://arxiv.org/html/2507.17131v1?ref=notatechguy.com) · [P5 — GitHub - facebookresearch/seamless\_interaction: Foundation Models and ](https://github.com/facebookresearch/seamless%5Finteraction?ref=notatechguy.com)*

## More from Not A Tech Guy

- [LoopX adds durable memory to Claude Code and Codex agents](https://www.notatechguy.com/loopx-adds-durable-memory-to-claude-code-and-codex-agents/)
- [GPT-6 Astra: OpenAI's first model to hit Critical cyber level](https://www.notatechguy.com/gpt-6-astra-openai-s-first-model-to-hit-critical-cyber-level/)
- [Google Fairwind Program: AI writes verified patches in minutes](https://www.notatechguy.com/google-fairwind-program-ai-writes-verified-patches-in-minutes/)

---

*Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.*