On 17 July 2026, researchers from Politecnico di Milano and LMU Munich posted an arXiv preprint describing AutoSynthesis, a multi-agent system that takes a research question in plain English and produces a complete meta-analysis [S1][P2]. The system screened more than 28 studies and extracted over 20 quantitative claims in its test run, with pooled effect estimates close to those produced by human experts [S1]. Whether that holds up outside a 28-study sample is the question every evidence reviewer now needs answered.

How the pipeline works

Meta-analysis is widely regarded as the strongest method for pooling evidence across studies, yet the workflow remains labour-intensive, time-consuming, and difficult to scale [S1]. A typical review takes months: researchers search databases, screen hundreds of papers by title and abstract, read full texts to decide which qualify, extract statistics by hand, compute effect sizes, and run statistical models. AutoSynthesis compresses that chain into a single automated pipeline [S1].

Feed it a research question in plain English and the system builds a search strategy, pulls relevant papers from scientific databases, filters candidates by screening, reads full texts to check eligibility, pulls out the quantitative statistics, converts them to standardised effect sizes, and runs a random-effects meta-analysis [S1]. It also runs heterogeneity analysis to check how effect sizes vary across subgroups, and performs risk-of-bias assessment to flag low-quality studies [S1]. The output is a report structured around PRISMA guidelines, the reporting standard for systematic reviews [S1].

The architecture uses multiple agents, each handling one stage of the pipeline. This mirrors the broader shift toward agentic AI, where specialised agents hand off to each other rather than relying on a single model to do everything. AutoSynthesis applies the same pattern to academic evidence synthesis.

What it means

For a reader with no background in evidence synthesis: meta-analysis is how science combines results from multiple studies to reach a stronger conclusion than any single study can. Think of it as averaging out the noise across dozens of experiments to find the signal. The problem is that doing one properly takes months of expert labour.

AutoSynthesis does not replace that expertise. The preprint describes a system that produces results similar to expert-conducted meta-analyses, not better ones [S1]. The pooled effect estimates, measured using Hedges' g (a standardised statistic for comparing effect sizes across studies), were close to those from manual reviews [S1]. But "similar" on 28 studies is a starting point, not proof of reliability at scale.

The real promise is speed and cost. If a system can draft a meta-analysis in hours instead of months, researchers can run more reviews, update them more frequently, and test more questions. That matters for fields like medicine and education policy, where evidence accumulates faster than humans can synthesise it.

What it means for business

A two-person consultancy that advises on health policy or education interventions could use a tool like this to produce evidence reviews for clients at a fraction of the current cost. The workflow changes on their desk: instead of spending six weeks screening papers, they spend a day reviewing the system's output, checking its study selections, and verifying its effect-size calculations.

For a pharmaceutical company running dozens of literature reviews for regulatory submissions, the appeal is obvious. But the risks are equally obvious. The preprint is not peer-reviewed [S1]. The evaluation covered only 28 studies [S1]. The degree of human oversight required is unclear from the preprint. Any operator adopting this kind of system would need to treat its output as a first draft, not a finished product.

The GitHub repository associated with the project shows 79 stars and 17 forks, with a release tagged 0.1.0.2 [P3]. That is early-stage adoption, not production deployment. A separate related repository, aascode/synthscholar, implements a PRISMA-aligned systematic review agent in Python [P5], suggesting the open-source ecosystem around automated evidence synthesis is growing but still nascent.

What we don't know yet

The preprint reports results on a single application with 28 studies [S1]. We do not know how AutoSynthesis performs on larger bodies of literature, on questions where evidence is contradictory, or on fields where study designs vary widely. The claim of similarity to expert meta-analyses lacks specific agreement metrics in the available text [S1].

We do not know how much human intervention the system requires. The preprint describes an end-to-end pipeline, but the degree of human oversight at each stage is not specified. Whether the system can handle qualitative evidence synthesis is settled: the authors specify quantitative only [S1].

The code's availability is partial. The GitHub repository exists [P3], but the preprint does not confirm whether the system is fully open-source or requires specific API access to underlying language models.

The next concrete event to watch is peer review. If this preprint survives journal review with its claims intact, and if independent teams replicate the results on larger samples, the case for automated meta-analysis strengthens. Until then, it is a promising demo on 28 studies.

If you want the peer-review verdict when it lands, subscribe and we will have it in your inbox.

Sources

More from Not A Tech Guy

Generated from an audited evidence pack with primary-source research. Social-media items are discussion signals, not verified facts. Nothing here is financial, legal or medical advice.