The strongest AI system tested on a new benchmark for scientific lineage reasoning reached only 27.3% exact accuracy — meaning the best large language model in the experiment got the lineage of scientific ideas wrong roughly three times out of four [S1]. The benchmark, called IdeaGene-Bench or IG-Bench, was posted as an unpeer-reviewed arXiv preprint on 10 July 2026 [S1], and it asks a deceptively simple question: can an AI follow the chain of inheritance that connects one scientific paper to the next? The answer, across 14 LLM-based scientist systems, is barely. What that failure exposes about the limits of AI-assisted research — and the specific bottleneck the authors say they've found — is the story worth reading for.

The genome analogy, decoded

The paper's central metaphor is that scientific ideas have genomes [S1]. Not literal DNA — the authors are explicit that this is an analogy — but a structured inheritance pattern. A new paper doesn't start from nothing. It inherits mechanisms from earlier work, patches known limitations, and recombines fragments of prior research, much the way a biological genome passes down genes, mutates them, and occasionally imports new material [S1].

The IdeaGene framework makes this concrete. Each paper or proposal gets broken down into a set of minimal, typed, evidence-grounded objects the authors call "Idea Genomes" [S1]. A tool called GenomeDiff then aligns these objects across papers and records what happened between them: inheritance, mutation, loss, external import, or novel insertion — six evolutionary dynamics in total [S1].

Think of it as a diff viewer for ideas. The same way a programmer can see exactly which lines changed between two versions of a file, GenomeDiff shows which conceptual building blocks a new paper carried forward, which it modified, and which it dropped entirely.

What the benchmark actually tests

IG-Bench contains 1,961 golden lineage traces — hand-curated records of how ideas actually evolved — along with 1,085 Idea Genome objects and 920 pairwise GenomeDiff records spanning 10 scientific domains [S1]. The code is public on GitHub under the VisionXLab/IdeasHaveGenomes repository, created 14 May 2026 [P4].

The benchmark splits into two evaluations [S1]:

  • IG-Exam — a closed-form test with 42 task types and 1,029 instances. It probes four skills: abstracting a paper into Idea Genome objects, tracing which components were inherited from where, reasoning about evolutionary changes, and verifying whether a claimed lineage actually holds [S1].
  • IG-Arena — a generation test. It asks whether a model can produce a research proposal that fits coherently as a descendant of a given lineage: inheriting the right Idea Genome objects, varying meaningfully from nearby work, and offering what the authors call "selection value" for future research [S1]. Arena uses a lineage-conditioned scoring metric called the Population-Evolution Score, or PES [S1].

The 27.3% number — and what breaks underneath it

The authors ran 14 LLM-based scientist systems through both evaluations [S1]. The headline result: the single best-performing system reached just 27.3% exact accuracy on lineage reasoning [S1]. That is not a passing grade by any standard — it's closer to a coin flip with a broken coin.

Two findings beneath that number matter more than the number itself.

First, the authors report that structured lineage context — giving a model the explicit inheritance chain rather than asking it to infer one — reshuffles system rankings rather than lifting all boats uniformly [S1]. Some models benefit from the extra context; others apparently don't, or even regress. That matters because it suggests the problem isn't simply "feed the model more context and it'll work." The ability to use lineage information is itself unevenly distributed across systems.

Second, the authors identify what they call a "compositional bottleneck" [S1] — the models can often identify individual components of an idea's lineage in isolation, but struggle to compose those components into a coherent chain of inheritance. They can see the genes; they can't read the genome.

The IG-Bench results land in familiar territory: models that can generate fluent scientific prose still falter when asked to reason about the structure of science itself.

What it means

Scientific progress is not a stream of isolated discoveries. It's a lineage — each paper standing on the shoulders of specific prior work, inheriting some methods, discarding others, occasionally importing a concept from a different field entirely. When a human researcher reads a new paper, they instinctively reconstruct this lineage: this builds on the transformer architecture, but swaps the attention mechanism for a sparse variant first proposed in 2021, and borrows the training objective from contrastive learning.

IG-Bench tests whether AI can do that reconstruction — and the answer is, right now, not well enough to rely on. A 27.3% accuracy rate means that if you ask an AI system to trace how a new idea descended from earlier work, it will get it wrong most of the time. For anyone hoping to deploy LLMs as research assistants that can survey a field, identify gaps, and propose genuinely novel contributions grounded in the existing literature, this is a flashing warning light.

The genome analogy is useful here because it reframes what "novelty" actually means in science. A genuinely new idea isn't one that appears from nowhere — it's one that recombines existing Idea Genome objects in a way no one has tried before, or imports an object from a different lineage. The IG-Arena evaluation tries to capture exactly this: a good proposal should inherit the right building blocks, diverge meaningfully from its nearest neighbours, and offer something future work can build on [S1]. That's a much richer definition of "novel" than "text the model hasn't generated before."

What it means for business

For the growing market of AI-powered research tools — literature review platforms, grant-writing assistants, automated survey generators — the IG-Bench results are a reality check. A two-person biotech startup using an LLM to scan the literature and identify "gaps" for their next project is operating on shaky ground if the model can't reliably trace which ideas descended from which. The model might surface a "gap" that's already been filled, or propose a "novel" combination that ignores a critical prior result.

More concretely: if your workflow involves asking an AI to summarise how a field evolved, identify the key lineage of a technique, or suggest where your work fits in the existing landscape, treat those outputs as starting points for human verification, not finished analysis. The compositional bottleneck the authors describe means the model may correctly identify individual connections but assemble them into a lineage that doesn't hold together.

For vendors building these tools, IG-Bench offers a ready-made evaluation harness — the code is public [P4] — that could expose whether their product actually understands scientific inheritance or merely produces confident-sounding text. A suburban research consultancy or a university lab support team could run the benchmark themselves to benchmark competing tools before committing to a subscription.

There's also a competitive angle. The finding that structured lineage context reshuffles system rankings [S1] means the model that performs best on standard benchmarks may not be the one that performs best when you give it the full inheritance chain. If your product depends on lineage reasoning, you need to test on lineage reasoning — not assume that general-purpose leaderboard performance transfers.

What we don't know yet

The paper is an arXiv preprint and has not been peer-reviewed [S1]. The benchmark design, the 27.3% figure, and the compositional bottleneck conclusion all derive from the authors' own evaluation on their own benchmark — which carries a risk of confirmation bias or overfitting to their specific test construction. No independent replication is cited.

Several questions remain open:

  • How does IG-Bench compare to existing lineage-reasoning tests? A separate GitHub project, fairydreaming/lineage-bench, was created in December 2024 to test LLM reasoning with lineage relationship quizzes [P5], and the Chain of Ideas agent project (CoI-Agent) has been on GitHub since October 2024 with 507 stars [P3]. The relationship between IG-Bench and these earlier efforts — whether they measure the same thing, and how their results compare — is not addressed in the evidence available.
  • Which 14 systems were tested, and how were they configured? The preprint describes the count but the evidence pack does not name the specific models or detail their configurations. Knowing whether the 27.3% figure belongs to a frontier model like GPT-class or Claude-class systems, or to smaller open-weight models, would materially change how to read the result.
  • Does the compositional bottleneck generalise? The authors identify it within their benchmark's task structure, but whether it reflects a fundamental architectural limitation of current LLMs or a quirk of IG-Bench's specific formulation is unclear without external validation.

The next concrete signal to watch: whether the benchmark is adopted by independent research groups, and whether the authors release the full per-system breakdown so the community can see which models handle lineage reasoning best — and whether any forthcoming model release closes the gap above 30%.

If you found this useful, subscribe to keep reading — we track the benchmarks and the breakthroughs, and we tell you what the numbers actually mean.

Sources: [S1] arXiv preprint, "Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation," arxiv.org, 10 July 2026. [P2] arXiv PDF (author list confirmed). [P4] GitHub: VisionXLab/IdeasHaveGenomes (benchmark code, created 14 May 2026). [P3] GitHub: DAMO-NLP-SG/CoI-Agent (related prior work, Chain of Ideas). [P5] GitHub: fairydreaming/lineage-bench (related lineage testing project, created December 2024).

Sources

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.