> ## Content Index > Fetch the complete content index at: https://www.notatechguy.com/llms.txt > Use this file to discover other available public pages before exploring further. # Amazon's HOPE prunes MoE experts 50%, gains 6.1% on coding - URL: https://www.notatechguy.com/amazon-s-hope-prunes-moe-experts-50-gains-6-1-on-coding/ - Published: 2026-11-10T11:39:32.000Z - Updated: 2026-11-10T11:39:32.000Z - Description: Amazon's HOPE method prunes half the experts in large MoE models by accounting for expert partnerships, gaining 6.1% on agentic coding tasks. - Author: Marcello Babbili - Tags: Technology & AI, AI Agents, AI Models, AI Chips Amazon researchers built HOPE, a method that prunes half the experts from large MoE models and still gains 6.1% on agentic coding over the next-best approach [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). The finding breaks the assumption behind every existing pruning method: that experts work alone. In reality, they form stable teams, and cutting them independently wastes performance [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). The memory problem HOPE targets is severe. Qwen3.5-122B, a model with 256 experts across 48 layers, carries roughly 250 GB of parameters. About 95% of that footprint is expert MLPs that sit idle for any given token, since only 8 of the 256 experts per layer actually fire [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). Every expert must occupy GPU memory at inference time regardless. **My read:** The 6.1% gain is narrow, limited to agentic coding at a 50% pruning rate, and the Qwen architecture stats come from a single source I cannot cross-check. But the core observation, that experts form stable cooperative groups, is consistent with independent work the authors cite, and the code is public on Hugging Face [P⁵](https://huggingface.co/Angshul/higher-order-pruner?ref=notatechguy.com). The number I would watch is whether HOPE's advantage holds at lower pruning rates, where most production deployments actually operate. ### The problem with solo scoring Existing pruning methods treat each expert as an island. They run a calibration dataset, assign every expert a scalar importance score, and cut the lowest scorers layer by layer [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). The authors call these first-order methods because they assume each expert's contribution is independent of all the others [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). That assumption breaks down. In their experiments, the authors observed consistent clusters of co-activated experts, groups that fire together reliably across tokens [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). Independent work has shown that this cooperation is a stable property of MoEs and matters for model performance [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). These co-selection patterns emerge early in training and stabilise throughout [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). Ignoring them leads to suboptimal pruning, especially when cutting aggressively [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). ### How HOPE scores expert pairs HOPE (Higher-Order Pruning of Experts) is a second-order method that scores experts by accounting for their pairwise interactions [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). Rather than asking how important expert A is alone, it asks how important expert A is given that it usually works alongside expert B. The method combines calibration-derived second-order information with a higher-order finite-difference estimate, according to the code repository on Hugging Face [P⁵](https://huggingface.co/Angshul/higher-order-pruner?ref=notatechguy.com). That code is available as a reusable Python API for pruning compatible unquantized PyTorch models. The repository does not include benchmark results or pruned checkpoints [P⁵](https://huggingface.co/Angshul/higher-order-pruner?ref=notatechguy.com). Pruning broadly avoids the latency problems that make dynamic loading or offloading of experts impractical, requires no training, and composes with quantization, though the source does not explicitly confirm HOPE preserves every one of these properties [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). ### What the numbers show At 50% pruning, HOPE achieves the best average rank across benchmarks of any method tested [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). On agentic coding tasks specifically, it gains up to 6.1% over the next-best pruning method [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). The authors do not report results at other pruning rates in the provided source, and the full comparison set of methods and benchmarks is not detailed. The paper, listed on arXiv by Alex M. Tseng, Prannay Kaul, Luca Zancato, Wei Xia, and Stefano Soatto of Amazon's AI Fundamental Research and AWS Agentic AI groups, is not peer-reviewed [P⁴](http://arxiv.org/abs/2609.18916v1?ref=notatechguy.com). The Hugging Face blog post, published November 9, 2026, is the clearest summary of the method [S¹](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com). For a team running a large MoE model on limited GPU memory, the practical question is whether HOPE's pairwise scoring preserves enough cooperative structure at the pruning rate they actually need. A team serving a 250 GB model on 80 GB GPUs might start with the public code [P⁵](https://huggingface.co/Angshul/higher-order-pruner?ref=notatechguy.com), prune at 50%, and compare agentic task performance against their current pruned checkpoint before committing. --- *Sources: [S1 — Your Experts Have Partners. There's HOPE for Keeping Them Together](https://huggingface.co/blog/amazon/higher-order-pruning-experts?ref=notatechguy.com) · [P2 — allenai/OLMoE](https://github.com/allenai/OLMoE?ref=notatechguy.com) · [P3 — Paper page - \`Keep it Together': Enforcing Cohesion in Extractive Summ](https://huggingface.co/papers/2402.10643?ref=notatechguy.com) · [P4 — Higher-order pruning of experts in mixture-of-experts language models](http://arxiv.org/abs/2609.18916v1?ref=notatechguy.com) · [P5 — Angshul/higher-order-pruner · Hugging Face](https://huggingface.co/Angshul/higher-order-pruner?ref=notatechguy.com)* --- *Written from 5 sourced items, 5 of them primary.* ![Qwen3.5-122B: experts per layer vs experts active per token](https://storage.ghost.io/c/6e/89/6e896869-22ef-4281-a213-b4c462c17cff/content/images/2026/11/chart_8e452f5a127d183e0d26.png) ## More from Not A Tech Guy - [AI agents face 284 known attacks, most defences untested](https://www.notatechguy.com/ai-agents-face-284-known-attacks-most-defences-untested/) - [ArtCraft open-source AI studio trends on GitHub with 40,263 stars](https://www.notatechguy.com/artcraft-open-source-ai-studio-trends-on-github-with-40-263-stars/) - [LLM jailbreak defense tunes attention heads without retraining](https://www.notatechguy.com/llm-jailbreak-defense-tunes-attention-heads-without-retraining/)