Amazon researchers built HOPE, a method that prunes half the experts from large MoE models and still gains 6.1% on agentic coding over the next-best approach S¹.
The finding breaks the assumption behind every existing pruning method: that experts work alone. In reality, they form stable teams, and cutting them independently wastes performance S¹.
The memory problem HOPE targets is severe. Qwen3.5-122B, a model with 256 experts across 48 layers, carries roughly 250 GB of parameters. About 95% of that footprint is expert MLPs that sit idle for any given token, since only 8 of the 256 experts per layer actually fire S¹.
Every expert must occupy GPU memory at inference time regardless.
My read: The 6.1% gain is narrow, limited to agentic coding at a 50% pruning rate, and the Qwen architecture stats come from a single source I cannot cross-check. But the core observation, that experts form stable cooperative groups, is consistent with independent work the authors cite, and the code is public on Hugging Face P⁵. The number I would watch is whether HOPE's advantage holds at lower pruning rates, where most production deployments actually operate.
The problem with solo scoring
Existing pruning methods treat each expert as an island. They run a calibration dataset, assign every expert a scalar importance score, and cut the lowest scorers layer by layer S¹. The authors call these first-order methods because they assume each expert's contribution is independent of all the others S¹.
That assumption breaks down. In their experiments, the authors observed consistent clusters of co-activated experts, groups that fire together reliably across tokens S¹. Independent work has shown that this cooperation is a stable property of MoEs and matters for model performance S¹. These co-selection patterns emerge early in training and stabilise throughout S¹. Ignoring them leads to suboptimal pruning, especially when cutting aggressively S¹.
How HOPE scores expert pairs
HOPE (Higher-Order Pruning of Experts) is a second-order method that scores experts by accounting for their pairwise interactions S¹. Rather than asking how important expert A is alone, it asks how important expert A is given that it usually works alongside expert B. The method combines calibration-derived second-order information with a higher-order finite-difference estimate, according to the code repository on Hugging Face P⁵.
That code is available as a reusable Python API for pruning compatible unquantized PyTorch models. The repository does not include benchmark results or pruned checkpoints P⁵. Pruning broadly avoids the latency problems that make dynamic loading or offloading of experts impractical, requires no training, and composes with quantization, though the source does not explicitly confirm HOPE preserves every one of these properties S¹.
What the numbers show
At 50% pruning, HOPE achieves the best average rank across benchmarks of any method tested S¹. On agentic coding tasks specifically, it gains up to 6.1% over the next-best pruning method S¹. The authors do not report results at other pruning rates in the provided source, and the full comparison set of methods and benchmarks is not detailed.
The paper, listed on arXiv by Alex M. Tseng, Prannay Kaul, Luca Zancato, Wei Xia, and Stefano Soatto of Amazon's AI Fundamental Research and AWS Agentic AI groups, is not peer-reviewed P⁴. The Hugging Face blog post, published November 9, 2026, is the clearest summary of the method S¹.
For a team running a large MoE model on limited GPU memory, the practical question is whether HOPE's pairwise scoring preserves enough cooperative structure at the pruning rate they actually need. A team serving a 250 GB model on 80 GB GPUs might start with the public code P⁵, prune at 50%, and compare agentic task performance against their current pruned checkpoint before committing.
Sources: S1 — Your Experts Have Partners. There's HOPE for Keeping Them Together · P2 — allenai/OLMoE · P3 — Paper page - `Keep it Together': Enforcing Cohesion in Extractive Summ · P4 — Higher-order pruning of experts in mixture-of-experts language models · P5 — Angshul/higher-order-pruner · Hugging Face
Written from 5 sourced items, 5 of them primary.
