A new arXiv paper shows that wrapping enterprise AI agents in code-owned "harnesses" stops recommendation and trace leaks at full utility — 120 out of 120 — while a bolt-on external guardrail doing the same job over-refuses so hard that utility collapses to 88/120 [S1]. Across 270 validation runs spanning three hosted models, the harness never let a broken contract through [S1]. The question the paper forces: if prompts alone can't guarantee safety, and guardrails trade safety for usefulness, what does the third option actually look like?

The prompt problem

Enterprise LLM applications typically begin as prototypes whose behavior is carried by prompts and retrieval context [S1]. That works in a demo. It falls apart in production, where you need source boundaries (which documents can the agent cite?), entity routing (which subsidiary is the user asking about?), answer contracts (what shape must the response take?), and reproducible traces (can you reconstruct what happened if a regulator asks?) [S1].

The paper's proposal is a "harness-engineering" approach that reconstructs exploratory prototypes into a traceable, auditable architecture [S1]. The core move: deterministic behaviour shifts out of the prompt and into code, manifests, schemas, and validation artifacts, all arranged around a replaceable composition boundary — the seam where the model plugs in [S1]. Source-backed claims remain the authority for runtime answers [S1]. In plain terms: the model generates, the harness checks, and only validated output reaches the user.

270 runs, zero escapes

The authors tested the approach on public data from five Korean corporate groups covering 25 listed companies [S1]. Across fixed validation scenarios, the harness preserved its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts [S1]. A fault-injection control — deliberately breaking contracts to see whether validators noticed — confirmed the harness flagged every violation [S1].

The headline number: across three hosted models, the harness passed all 270 composition-boundary runs [S1]. When models were swapped, failures stayed confined to the model-composed side and were caught and recorded [S1]. Nothing leaked through.

The ablation that settles the argument

The paper's most revealing test holds the model fixed and varies only the enforcement layer [S1]. With prompt instructions alone — no code-owned checks — recommendation-language violations and internal-trace leakage reached the reader [S1]. The model was told not to do these things. It did them anyway.

The harness blocked both violation types entirely [S1]. A bolt-on external guardrail also blocked them — but over-refused, dropping utility to 88 out of 120 [S1]. The harness held full utility at 120/120 [S1]. In this ablation, only code-owned enforcement preserved both safety and utility [S1].

The gap is stark. The guardrail approach treats the model as untrustworthy and filters everything, including legitimate responses. The harness approach treats the model as a replaceable component and enforces contracts in code around it — so safety doesn't cost you usefulness.

What it means

The paper articulates something practitioners have been groping toward: prompts are not contracts. You cannot audit a prompt. You cannot version-control a prompt's effect on output. You cannot guarantee that swapping one model for another won't break a prompt's implicit rules. Harness engineering moves the guarantees that matter — source attribution, entity routing, output shape, trace reproducibility — into artifacts you can inspect, version, and test [S1].

The deep-research trail backs this up: a related GitHub project describes "Agentic Harness Engineering" as observability-driven evolution of coding-agent harnesses, reporting 84.7% pass@1 on Terminal-Bench 2 [P3]. A separate arXiv paper, "Auditing Agent Harness Safety," signals that harness safety is becoming its own research area [P4]. Open-source implementations are already appearing — one TypeScript harness-engineering repository has 16 stars and 78 open issues, suggesting real adoption and real rough edges [P2].

What it means for business

For a two-person consultancy running an AI agent to answer client queries about listed companies, the paper's pattern offers a concrete shift. Instead of hoping the model won't recommend a stock when it shouldn't, you write a validator that rejects any output containing recommendation language — and you log every rejection [S1]. When a client asks about the wrong subsidiary, an entity-routing schema catches it before the response ships [S1].

The replaceable composition boundary matters operationally: you can swap model vendors without re-auditing the entire pipeline, because the guarantees live in code, not in the model's tendencies [S1]. The over-refusal problem the paper documents is equally concrete — a bolt-on guardrail that drops utility to 88/120 is an agent that refuses roughly a quarter of legitimate queries, which means frustrated users and lost work [S1]. The harness approach keeps those queries flowing while still blocking the violations that get you in trouble.

For compliance teams, the reproducible-trace requirement is the unlock: every run produces a versioned record of what source was cited, what entity was routed, and what validators fired [S1]. That's the difference between "we think the agent behaved" and "here is the proof."

What we don't know yet

The evaluation is narrow. The harness was tested on a single public-data slice of Korean corporate groups — 25 listed companies across five conglomerates [S1]. Whether the pattern holds for other domains, languages, or data structures is unproven.

The paper is a preprint on arXiv; it has not been peer-reviewed, and all claims derive from the authors' own abstract [S1]. The three hosted models are unnamed, limiting reproducibility. The "bolt-on external guardrail" used for comparison is not identified, so the 88/120 figure is an abstract benchmark, not a test against a specific commercial product [S1].

No evidence supports that the approach has been deployed in a live production environment, or that it eliminates hallucinations beyond the specific contract violations tested [S1]. The next concrete signal to watch: whether independent teams replicate the 270-run result on non-Korean corporate data, and whether the open-source harness repositories now maturing on GitHub [P2] begin publishing their own validation suites.

If your work touches AI agents and you want the decode on what actually moves the needle — not the vendor pitch — the next story is already in the pipeline. Subscribe and you won't miss it.

Sources

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.