AI-written rubrics for grading LLM research agents are biased toward high scores, overly fine-grained, and less adaptive to different paper domains, according to a preprint posted on arXiv on 15 July [S1]. The work also tested whether those flaws can be fixed, and the answer is a qualified yes, with the right setup pushing AI-generated rubrics close to human-level grading. What that setup is, and what it costs, changes how seriously we should take automated evaluation of AI agents.

Why rubrics matter for AI agents

LLM-based research agents are increasingly tasked with reproducing scientific papers: taking a published study and generating the code to replicate its experiments. To grade how well the agent did, you need a rubric, a structured set of criteria that defines what a correct reproduction looks like.

The authors argue that direct paper-to-repository comparison, just asking an LLM to compare the paper against the generated code, is prone to hallucination [S1]. Rubric-based evaluation, where you break the task into specific checkable criteria, is more reliable. But building paper-specific rubrics by hand requires substantial expert effort, which limits how many papers you can benchmark. They cite PaperBench as an example of a benchmark that hits this scalability wall [S1].

So the question becomes: can LLMs write their own rubrics, well enough to grade other LLMs?

What the study did

The authors, Hanhua Hong, Yizhi Li, Jiaoyan Chen, and colleagues at the University of Manchester and A*STAR's Institute for Infocomm Research [P4], reformulated rubrics into a checklist-style format [S1]. They tested four generation settings across two backbone models [S1], then evaluated the resulting rubrics two ways: intrinsically, by measuring semantic similarity to human-written rubrics, and extrinsically, by checking whether the AI rubrics produced scores that aligned with ground-truth scores [S1].

The paper is an arXiv preprint and has not been peer-reviewed [S1]. It also appears on OpenReview as an ACL ARR 2026 May submission [P2], meaning it is under review but not yet accepted.

Three things that go wrong

Out of the box, LLM-generated rubrics have three systematic problems.

First, they are overly fine-grained [S1]. Where a human grader might write "correctly implements the loss function," an LLM tends to split that into a dozen sub-criteria, each checking a minor detail. More granularity sounds more thorough, but it makes the rubric brittle: a correct implementation that doesn't match the exact sub-criteria gets penalised.

Second, the generated rubrics skew toward high scores [S1]. When used to grade an agent's output, they tend to give generous marks. For a field already worried about AI self-evaluation being too lenient, this is a warning sign.

Third, they adapt poorly to different paper domains [S1]. A rubric for a computer vision paper and one for a natural language processing paper should look different. The LLM-generated versions tend to be generic, missing the domain-specific criteria that matter most.

What it means

The core finding is not that LLMs cannot write rubrics. It is that they cannot write them without help. The authors report that augmented settings, where the generation process is supplemented with additional information or constraints, produce scores that line up much better with human scores, with the strongest setting getting close to what human graders achieve [S1].

But the intrinsic gains are more modest [S1]. In plain terms: the augmented rubrics produce scores that align better with human scores, but the rubrics themselves still don't look much like what a human would write. They work, but they don't read like expert rubrics. That gap matters because it means the improvement is fragile. It depends on the augmentation, not on the model suddenly understanding what a good rubric is.

This connects to a broader pattern in AI evaluation research: a rubric that produces the right score is not the same as a rubric that captures the right reasoning.

What it means for business

For teams building or buying AI agent evaluation pipelines, the takeaway is concrete. If you are using an LLM to generate grading rubrics for open-ended agent outputs (code reproduction, report generation, research tasks), do not trust the raw output. Plan for an augmentation step, such as feeding the model example rubrics or domain-specific context before it generates criteria.

A two-person AI startup building an agent evaluation tool should budget for human review of generated rubrics, at least initially. The study suggests the augmented settings get close to human-level scoring, but "approaching" is not "meeting," and the abstract provides no quantitative metrics or confidence intervals to say how close [S1].

For larger organisations running benchmarks at scale, the checklist-style format the authors adopted may reduce the bias toward high scores by forcing a binary judgment rather than a generous one. Simple yes/no criteria leave less room for a model to talk itself into a high mark.

What we don't know yet

The preprint does not name the two backbone models tested [S1], so we cannot tell whether the results generalise across model families or are specific to certain architectures. The "first systematic meta-evaluation" claim is self-attributed [S1] and has not been verified against the broader literature. A related preprint, RubricBench [P3], addresses similar questions about aligning model-generated rubrics with human standards, suggesting the field is more active than the "first" claim implies.

The paper is under review at ACL ARR 2026 [P2]. Review decisions and any revisions will clarify whether the augmented settings hold up under peer scrutiny, and whether the authors release the specific augmentation methods in enough detail to replicate.

No quantitative metrics appear in the abstract [S1], so the magnitude of improvement, how much "substantially" means and how close "approaching" is, remains unclear until the full paper is examined or the authors release code.

The next concrete event is the ACL ARR 2026 review cycle, which should produce decisions and revisions in the coming months. If you are building or buying AI agent evaluation, the details here will keep shifting. Subscribe to follow what happens next.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.