Zixi Chen and Andrew Gordon Wilson at New York University, with colleagues at Q Labs, report that a 7.4-billion-parameter architecture matches GPT-3's 13-billion-parameter model on the CORE benchmark using roughly 20 times less compute S¹. That result, if it holds, challenges a core assumption of the scaling-laws literature: that the exponent governing how efficiently a model improves with more compute is fixed by data and model size, not by how the model is wired S¹.

My read: This is the first paper I've seen that claims architectural changes can shift the scaling exponent itself, rather than just moving you along the same curve. I don't buy the "exponential improvements" framing yet, because the authors use that term loosely and the preprint has no peer review and no external validation. The 20× compute claim is limited to one benchmark (CORE) and one comparison point (GPT-3 13B), so I'd treat it as a directional signal, not a production-ready result.

What the authors actually changed

The team, based at NYU and Q Labs P³, tested three architectural interventions on transformer pre-training. First, model growth: starting with a small network and expanding it during training, either with shared weights across the expanded sections or without. Second, recursion via looped transformers, running the same transformer blocks multiple times to deepen the network without adding parameters. Third, a boundary operator that normalizes and injects an earlier block's output into a later one S¹.

Model growth, with and without shared weights, produced the biggest shifts in the scaling exponent S¹.

The boundary operator also improved compute efficiency in a vanilla transformer, though less dramatically S¹. Standard looping, where the same blocks are applied repeatedly, had a different effect: in data-constrained, multi-epoch training, where the model sees the same data multiple times, looping acted as a regularizer, and it was compute-optimal to increase the number of loops as scale grew S¹.

Computational depth ties these findings together: for a fixed compute budget, increasing the usable depth of the transformer yields efficiency gains that grow with scale S¹. This connects to broader work on recursive approaches. Google DeepMind researcher Ibrahim Alabdulmohsin has explored recursive inference scaling for language and multimodal systems P⁴, and the open-source RecursiveMAS project on GitHub, created in April 2026 with 927 stars, applies recursive algorithms to multi-agent collaboration P⁵. As we found when AI interaction created behaviour no single model showed alone, recursive and compositional structures can produce capabilities that emerge from interaction rather than raw scale.

The 20× claim and its limits

The headline number: a 7.4-billion-parameter model growth architecture matches GPT-3 13B on the CORE benchmark with roughly 20× less compute S¹.

Parameter count: model growth architecture vs GPT-3

That is a single benchmark, not a sweep across evaluation suites. The paper reports no error bars on the compute comparison, and the 7.4B model's performance on other benchmarks is not claimed to match GPT-3 13B.

The preprint, submitted to arXiv on 16 September 2026, has not been peer-reviewed S¹. The authors base their claims on their own experiments with no external corroboration. The distinction between data-constrained and data-rich settings matters: the finding that looping is compute-optimal applies specifically to the multi-epoch, data-constrained regime, and the authors do not claim it holds when training data is abundant S¹.

This matters because, as we found when tokenizer metrics predicted model performance, the relationship between architecture and capability is not always intuitive. And as we saw with federated learning privacy error scaling, a mathematical improvement in one component can change the economics of the whole pipeline.

Who would use this first

A lab training models in a data-constrained regime, where fresh text is scarce and multi-epoch training is unavoidable, has the most to gain. The finding that increasing loops with scale is compute-optimal in that setting suggests a practical recipe: rather than scaling parameters alone, increase the number of times the same blocks process each token. The boundary operator, which requires only normalising and injecting an earlier block, is the cheapest intervention to test on an existing architecture.

For teams running inference at scale, the computational-depth framing connects to the broader push for recursive inference that Google DeepMind has explored P⁴.

The open-source RecursiveMAS project P⁵, with its MIT licence and Python implementation, shows the same idea spreading beyond single-model training into multi-agent systems. The next checkpoint for this work is peer review and independent reproduction. The preprint is available on arXiv S¹, and the authors have not released code as of the submission date.

Sources: S1 — How Model Growth, Recursion, and Boundary Operators Influence Scaling · P2 — [2609.19107] How Model Growth, Recursion, and Boundary Operators Influ · P3 — How Model Growth, Recursion, and Boundary Operators Influence Scaling · P4 — Recursive Inference Scaling:A Winning Path to Scalable Inference in La · P5 — RecursiveMAS/RecursiveMAS

Written from 5 sourced items, 4 of them primary.