Aayush Marasini and Barsat Khadka at the University of Southern Mississippi tested whether AI can edit a neural network's weights directly, and the best model scored 53.5% S¹. That score, on a 42-task benchmark built with Nick Rahimi, is above chance but leaves nearly half the work undone, and no one outside their lab has verified it S¹. The idea it tests, weight-space programming, challenges the assumption that neural networks are black boxes only gradient descent can tune.
My read: The 0.535 score is the number I would watch, but I would not treat the model ranking as settled. GPT-6.1-sol scored 0.568 but only ran on 38 of the 42 tasks, while the top Claude model's 0.535 covers all 42, so the headline ranking depends on which subset you accept. The model versions named in the paper do not match any commercial product I can identify, which means these results are self-reported against versions nobody outside the authors' lab has checked. The claim that agents can produce working control policies without executing them is the one I am most skeptical of, because the paper reports no error bars and no third-party audit.
What weight-space programming actually means
Neural networks learn by gradient descent, a process that nudges millions of internal numbers, called weights or parameters, through thousands of rounds of trial and error until the network produces the right output. Weight-space programming asks a different question: can a language model look at those finished numbers and reason about them directly, the way a programmer reads source code S¹? The tasks in this benchmark cover five tracks: reading a network to figure out what it was trained to do, fixing one that fails, writing a working network from scratch, recognising when an edit is mathematically impossible, and compressing a network without any training data S¹.
The networks in the test range from 2,000-parameter control policies, the kind that steer a simple robot, to 7-billion-parameter language models S¹. That spread matters. A 2,000-parameter network has few enough numbers that a language model can hold them all in its input and reason about each one. At the other end, a 7-billion-parameter model has millions of times more weights, and no current language model can hold all of them in a single prompt.
What the scores reveal
The authors ran several models through all 42 tasks and reported a macro score, a simple average across all tracks. Claude Opus 5.5 reached 0.535 and DeepSeek v4-flash managed 0.215 S¹, with two other models between those poles S¹. On a 38-task subset, GPT-6.1-sol scored 0.568 against the same Claude model's 0.546 on those tasks S¹. These are self-reported figures from a preprint that has not been peer-reviewed, and no one outside the authors' lab has reproduced them.

The pattern across tracks is more telling than the overall number. The authors report that frontier agents can already inspect small networks and produce working control policies without executing them S¹. But the same agents cannot yet repair language models, shrink them without training data, or distinguish an impossible edit from a difficult one S¹. The easy end of WSP, inspecting and building small networks, is starting to work. The hard end, surgery on large models, is not.
Why open weights change the picture
The benchmark lands as open-weight models are spreading fast. NVIDIA released four open-weight Cosmos 3 world models earlier this month, and the company joined an EU science initiative with 250 open models in October. When model weights are public, anyone can download them, and WSP is, in theory, a way to modify or repair those weights without the expensive retraining that usually requires. A separate project called openwsp, created on November 3, is already building open tooling for this kind of weight-level editing P⁵.
The practical question is whether any of this is reliable enough to use. A model that can read a 2,000-parameter control policy and tell you what it does could help a robotics engineer debug a failing controller without retraining it.
A model that claims to patch a 7-billion-parameter language model but cannot distinguish a fixable bug from an impossible one is a liability. The day before this preprint appeared, separate research found that AI models bend advice to serve the boss's bonus, a reminder that models operating on other models inherit whatever blind spots the outer model carries.
What a robotics lab should check first
A team working with small control policies, the 2,000-parameter end of the spectrum, is the most natural first user. The benchmark's code, task registry, OS-level sandbox harness, and live leaderboard are on GitHub under an MIT licence S¹ P². The repository was created on November 13 and had zero stars at the time of research P², so it is brand new. A robotics engineer could pull the repo, run the reading and writing tracks on their own control policies, and check whether the model's output matches the network's actual behaviour. The signal to stop: if the model cannot correctly identify what a 2,000-parameter policy does, it is not ready for anything larger.
The next checkpoint is peer review. The paper has not been through it, and the authors' scores could change under scrutiny. The live leaderboard on GitHub is the place to watch for independent reproductions.
Sources: S1 — WeightBench: Evaluating Language Models as Weight-Space Programmers · P2 — Aayush-Marasini/WeightBench · P3 — huggingface/lm-evaluation-harness · P4 — WeightBench: Evaluating Language Models as Weight-Space Programmers · P5 — openwsp/openwsp
Related reading
- NVIDIA releases four open-weight Cosmos 3 world models — our technology desk, 2026-11-06
- AI models bend advice to serve the boss's bonus — our technology desk, 2026-11-13
- NVIDIA joins EU science initiative with 250 open models — our technology desk, 2026-10-18
Written from 5 sourced items, 4 of them primary.