A new benchmark built from 7,500 dialogue traces across 50 detailed personal profiles shows that AI agents trained to mimic individual users leak private attributes, communication styles, and personality traits, and the defenses designed to stop it don't generalise S¹. The preprint, posted on arXiv on 4 August, introduces AntiSkillBench, an end-to-end test for privacy risks in "persona skills", the technique of compressing a person's interaction history into a portable artifact an agent can reuse S¹. What happens when that artifact carries your voice and your habits into a system you don't control?

My read: This is the first benchmark I've seen that treats persona skills as a distinct privacy threat rather than a convenience feature. The finding that risks persist across three frontier agents and three distillation strategies tells me the problem is structural, not a bug in one model. I don't buy the defense claims in the abstract, because the authors themselves say existing defenses only work in narrow settings tied to specific distillation methods and break down when applied more broadly S¹. That is an admission that nobody has a working fix yet.

How persona skills concentrate what you'd rather keep scattered

Persona skills work by distilling your past interactions into a compact, reusable package S¹. The way you phrase things and the tools you reach for get compressed into an artifact a downstream agent can load and act on S¹. The convenience is obvious. The risk is that this process takes fragments of personal data that might be harmless individually and concentrates them into a single artifact that reveals far more in combination S¹.

The benchmark makes this point explicitly: the pipeline gathers scattered personal signals and, by reusing them repeatedly, magnifies what they reveal S¹. A single chat log might reveal your first name. A persona skill built from a thousand of your interactions can reveal how you negotiate, what you avoid, and what that pattern says about you.

What AntiSkillBench actually measures

The benchmark has two layers. At the skill level, it tests whether the distilled persona artifact itself leaks private information S¹. At the agent level, it measures attribute disclosure (does the agent reveal your personal details when prompted?) and behavioral impersonation (can an attacker reproduce your communication style?) S¹.

The evaluation covers three skill-distillation strategies and tests across three frontier AI agents, though the preprint does not name them S¹. The defense side tests four configurations, split between online interventions (active risk suppression during generation) and post-hoc interventions (passive provenance protection after the fact) S¹.

The dataset is 7,500 persona-grounded dialogue traces built from 50 behaviorally rich profiles spanning diverse task scenarios S¹. That is a small profile count but a deep per-profile trace set, roughly 150 dialogues per person, which matters because the threat is about pattern accumulation, not single-record exposure.

The defenses don't hold

Here is the finding that should make anyone building persona-based agents pause. Existing defenses show limited and distillation-dependent effectiveness against persona-skill risks S¹. They fail to generalise across risk types and distillation strategies S¹. A defense that works against attribute leakage in one distillation method may do nothing against behavioral impersonation in another.

The risks extend beyond obvious personal data. The benchmark shows leakage from explicit attributes into communication styles and personality traits S¹. An attacker who extracts your persona skill might not need to read your address. They could learn to sound like you well enough to fool someone who knows you.

This connects to a wider pattern in the field. IMPersona, presented at COLM 2025, separately evaluated individual-level LLM impersonation, finding that models can mimic stylistic and contextual features of specific people P². AgentLeak, an open benchmark on GitHub, tests privacy leakage across seven channels in multi-agent systems P⁵. MCP-Persona, another recent preprint, benchmarks LLM agents on real-world personal applications via environment simulation P⁴. The field is circling the same problem from different angles: personalisation and privacy are on a collision course.

AntiSkillBench adds a harder question to this body of work: what if the skill itself is the leak?

What to do about it

If you are building or buying agent systems that personalise to individual users, the takeaway is that you cannot assume existing privacy controls transfer. A defense built for retrieval-based memory, where you can redact a stored record, may be useless against a distilled skill where the personal signal is baked into the model's behavior S¹.

Consider a legal tech startup building a contract-review agent that learns each client's negotiation style. The persona skill captures how a particular general counsel pushes back on indemnity clauses, which jurisdictions they prefer, and how they frame risk. If that skill leaks, a competitor could reproduce that GC's approach or extract their negotiation strategy. The startup needs to treat the persona artifact as a sensitive asset in its own right, not as a byproduct of personalisation.

One practical step this week: audit any persona or personalisation feature in your agent stack and ask whether the distilled artifact can be queried for the source attributes it was built from. If it can, you have a leakage path that current defenses may not cover.

What we don't know yet

The preprint is not peer-reviewed, and the three frontier agents tested are not named in the abstract S¹. We don't know which models are vulnerable, how severe the leakage is in quantitative terms, or whether any of the four defense configurations reduced risk meaningfully for any specific distillation strategy. The abstract says defenses are "limited" but does not attach numbers to that claim in the material available.

We also don't know how persona skills behave in production conditions. The 50 profiles in AntiSkillBench are rich but synthetic. Real users generate messier, more identifying data over longer periods.

The next signal: the peer-review process for this preprint, likely through a venue such as COLM or NeurIPS given the topic and related work in the space P². We'll check the claims against any accepted version or reviewer comments when they surface.

If you want that follow-up, subscribe now.

Sources: S1 — When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonati · P2 — IMPersona: Evaluating Individual Level LLM Impersonation | OpenReview · P3 — ByteDance/InfiniteYou · Hugging Face · P4 — MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applicatio · P5 — Privatris/AgentLeak

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.

AntiSkillBench scale at a glance