> ## Content Index
> Fetch the complete content index at: https://www.notatechguy.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Hugging Face harness adapters lift Qwen 27B score 39% to 55%
- URL: https://www.notatechguy.com/hugging-face-harness-adapters-lift-qwen-27b-score-39-to-55/
- Published: 2026-11-21T10:57:25.000Z
- Updated: 2026-11-21T10:57:27.000Z
- Description: Hugging Face's harness adapters train a LoRA inside the agent's own runtime, lifting Qwen3.8-27B from 39% to 55% on Terminal-Bench while halving token
- Author: Marcello Babbili
- Tags: Technology & AI, Google, AI Agents

Hugging Face shipped harness adapters on November 20, training a small LoRA inside the agent's actual runtime rather than a re-implementation of it [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). On Qwen3.8-27B running the Pi harness against Terminal-Bench 2.1, accuracy climbed from 39% to 55% pass@1 while average token use fell from 2.9K to 1.5K per task [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com) — figures that are Hugging Face's own, with no error bars reported and no evaluation code released for outside reproduction. Most RL frameworks train against a copy of the agent loop, not the harness that ships to production [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). The model optimizes for a simulator, then encounters a different system when deployed.

Hugging Face's protocol executes the harness as delivered, placing a proxy between the harness and vLLM that captures the exact tokens generated for the trainer [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). That proxy is the core mechanism: the model tunes against the same tool calls, prompt formatting, and failure modes it will encounter in production.

**My read:** The 16-point jump is striking but entirely self-reported, with no independent corroboration [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). I would watch whether the merged "super adapter" actually matches the individual adapters' scores, because the launch post only says it runs unchanged under all three harnesses, not that it performs as well. The token halving is the number that matters for inference bills, but the title's "cost" promise rests on token efficiency, not demonstrated pricing savings.

![Qwen3.8-27B Terminal-Bench 2.1 pass@1, Pi harness](https://storage.ghost.io/c/6e/89/6e896869-22ef-4281-a213-b4c462c17cff/content/images/2026/11/chart_c966027309eabe9210df.png)

### Adapters train inside each harness's live runtime

The team trained one LoRA adapter per harness for Qwen3.8-27B, using reinforcement learning inside each harness itself [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). The three harnesses were Pi, OpenCode, and Hermes Agent, tested on harder real-world software engineering tasks than the data-analysis and code-building work in prior experiments [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). Training, datasets, and agent rollouts ran on the Hub using TRL's AsyncGRPOTrainer, OpenEnv, and harbor [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). The base model's weights stayed frozen. Only the LoRA adapter changed.

According to Hugging Face, most of the gain came from the model reasoning less, calling fewer tools, and getting it right on the first attempt [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). Each individual adapter beat the base model under its own harness on both performance and efficiency [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). The three LoRAs were also merged into a single adapter that runs unchanged under Pi, OpenCode, and Hermes Agent, though the launch post does not claim the merged version matches the individual adapters' scores [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com).

### Public artifacts include adapters, Docker images, and a tool-call inspector

Hugging Face released all artifacts through a public collection: the adapters, GGUF files, RL tasks, and a safeagents plugin that inspects every tool call before it runs [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). Ready-to-use Docker images come with the target harnesses pre-installed, using Pi 1.5.2, OpenCode 2.2.7, and Hermes Agent 0.22.6 [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com).

A team running Qwen3.8-27B inside Pi on a coding workload could pull the Pi-specific adapter from the collection, load it alongside the frozen base model, and compare pass@1 and token use on their own task distribution against the unadapted baseline. The safeagents plugin adds a guardrail layer, checking each tool call before execution [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com).

No one outside Hugging Face has independently verified the benchmark numbers. The results cover only the 27B parameter scale and only three harnesses, and the launch post's title promises cost improvements but demonstrates token efficiency rather than pricing data [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). Related approaches exist in the same space: Agent Lightning v1.0 keeps the deploy-time harness in the loop through a proxy, an approach its authors call "harnessed agentic RL" [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com), and SkyRL's Harbor integration trains on Harbor trials by converting chat histories into tokens for the trainer [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com). The broader push to cut RL training costs is visible in work like NgramRL's 7x rollout reduction.

The Hugging Face collection is live now. The adapters were trained on a custom agentic coding dataset built with the Hub, OpenEnv, and Repo2RLEnv [S¹](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com), and performance on tasks outside that distribution remains untested.

---

*Sources: [S1 — Introducing harness adapters: adapt your favorite model to your harnes](https://huggingface.co/blog/harness-adapters-at-scale?ref=notatechguy.com) · [P2 — Harness adapters: Safe RL for efficient agentic development](https://huggingface.co/blog/harness-hub/harness-adapters?ref=notatechguy.com) · [P3 — QwenLM/Qwen3](https://github.com/qwenLM/qwen3/?ref=notatechguy.com) · [P4 — Hugging Face – Blog](https://huggingface.co/blog?ref=notatechguy.com) · [P5 — huggingface/transformers](https://github.com/huggingface/transformers/?ref=notatechguy.com)*

---

*Written from 5 sourced items, 5 of them primary.*

## More from Not A Tech Guy

- [Google's Duplex now calls businesses for you inside Gemini](https://www.notatechguy.com/google-s-duplex-now-calls-businesses-for-you-inside-gemini/)
- [Google Gemini imports chat history from rival AI apps](https://www.notatechguy.com/google-gemini-imports-chat-history-from-rival-ai-apps/)
- [Google's 105-qubit quantum computer wins an unwinnable game](https://www.notatechguy.com/google-s-105-qubit-quantum-computer-wins-an-unwinnable-game/)