arXiv listed a preprint on November 3 showing that a de-identification editor which writes the wrong sex into medical records ranks first in three common privacy scores S¹. A fourth scoring method ranks the same editor sixth of eight, and a rule-aware adversary still recovers the attribute from the same text S¹. The disagreement across scoring methods exposes how much of a privacy verdict depends on the test setup rather than the text.

My read: The gap between first and sixth place for the same editor across scoring rules is the finding that grabs me. If an editor that actively lies about a patient's sex can top three privacy rankings, those rankings are measuring something other than privacy. The number I would watch is the relabelling-invariant AUC's sixth-of-eight result, because the disagreement across methods is the real signal. I don't buy that this generalises beyond MedQA yet, since that's the only named corpus in the stress test, and the paper is not peer-reviewed.

How the test works and where it breaks

The standard way to check whether de-identified text still leaks private information is to give it to a language model acting as an adversary, and ask the model to guess a protected attribute about the person described, such as their sex S¹. If the adversary guesses correctly more often than chance, the text is leaking. The authors introduce a new score they call a prior-normalised clipped leakage score, which measures how much the adversary's confidence improves on the correct answer compared to a baseline guess S¹.

Every privacy verdict, the authors argue, depends on what they call the evaluation interface: which person is queried, what answer options are given, what the prior probabilities are, which adversary model is used, and which scoring rule is applied S¹.

Change any of those, and the verdict can flip.

The authors built a traceable editor called SACC, which lets them ablate two editing principles and show where each one's advantage does not hold S¹. In their stress test, they created an editor that writes the opposite-sex form at exactly the same spans a redaction editor would redact. Where a redaction tool removes gendered pronouns, this editor swaps one for the other.

This opposite-sex editor misleads the adversary about most people S¹. Yet it ranks first among editors scored on the same MedQA entries under the prior-normalised clipped leakage score S¹. It also ranks first under attack accuracy and under proper scoring rules, which measure how well-calibrated the adversary's guesses are, read as privacy S¹. A relabelling-invariant reading of AUC, which adjusts for label swaps, ranks that same editor sixth of eight S¹.

Opposite-sex editor ranking under four privacy scoring methods lower is better, of 8

Three scoring methods say the lying editor is the most private.

A fourth says it is below average, and a rule-aware adversary, one that knows the swap rule, recovers the attribute from the same text anyway S¹.

What changes when you vary the interface

On ordinary releases, most verdict changes are gains or losses of separability. The text becomes more or less distinguishable between attribute values, rather than genuinely more or less private S¹. Of six interface choices varied over all pairs, only the adversary model reverses comparisons, accounting for 5 of its 124 changes S¹. The query can also target the wrong person, another way the interface skews the verdict S¹.

This connects to a broader pattern in privacy research. As Apple researchers found when plugging a privacy leak in private AI training, the gap between what a privacy mechanism claims and what an adversary can extract is where the real risk lives. Related work on textual sanitization has warned that surface-level redaction leaves deeper signals intact P², and Pin-Yu Chen of IBM Research and colleagues have shown that inference-time leakage in neural networks can expose attributes even when the model never sees the raw text P³. The method you choose to evaluate a text system can matter as much as the system itself, as AI text detection showed when it met graph theory in a separate preprint last month.

When we ran the released demo from the OpenMed project, a related tool that claims to adversarially scan de-identified clinical text for residual identifiers P⁴, the script crashed within 12 seconds with a missing Python dependency: ModuleNotFoundError: No module named 'yaml' [LAB1]. The demo would not run on our CPU-only box without manual dependency installation. This is third-party code, not the paper's own release, so the crash tells us nothing about the paper's claims. It only shows that the surrounding tooling is not yet plug-and-play.

A separate GitHub repository for privacy auditing of language models, by the user kiddyboots216, has 2 stars and an Apache 2.0 licence P⁵. It contains Python and Shell code but, with 2 stars, has not seen broad adoption.

Who should care

A hospital data team preparing a de-identified corpus for research release would be the first to feel this. If their privacy audit uses attack accuracy or a proper scoring rule, the authors' stress test suggests the audit could rank a deliberately misleading editor as the most private option. The practical check: run the same text through a relabelling-invariant AUC score as well. If the rankings disagree, the evaluation interface is doing more work than the de-identification.

The paper is not peer-reviewed, and MedQA is the only named dataset in the stress test S¹. The authors vary the evaluation interface on fixed releases of three corpora, but the stress test results are reported on MedQA entries specifically S¹. Generalising to other domains, such as legal text or financial records, is not yet supported by the evidence.

The preprint appeared on arXiv on November 3, 2026, in the cs.CR and q-fin.GN categories S¹. No peer review date or venue acceptance is listed.

Sources: S1 — Residual Attribute Leakage Is an Interface: Auditing Privacy Evaluatio · P2 — A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond · P3 — PEEL the Layers and Find Yourself: Revisiting Inference-time Data Leak · P4 — skills/auditing-deid-leakage/SKILL.md · P5 — kiddyboots216/privacy-auditing-of-llms · LAB1 — Not A Tech Guy code lab: sandboxed run of python examples/v30_clinical

Written from 6 sourced items, 4 of them primary, plus a sandbox run of the released code.