NVIDIA released Switchyard, an open-source Rust proxy that routes AI agent requests across multiple model providers, and the repo reached GitHub's daily trending list on 18 November 2026 S¹. The tool translates between OpenAI and Anthropic APIs on the fly, so a team running Claude Code or Codex can point those agents at cheaper open-weight models without changing application code S¹P⁴.
Switchyard embeds into existing gateways and supplies only routing decisions
Switchyard's core is a Rust library called libsy, designed to sit inside a gateway or agent harness you already run S¹. You keep your own model clients and credentials. Switchyard supplies only the routing decision, picking which model handles each request by mixing cheaper options with stronger ones to trade off task success, cost and latency S¹. No independent benchmarks with error bars have been published for these routing decisions; the capability claims rest on NVIDIA's own documentation and README S¹.
NVIDIA also ships a self-hosted Agent Router that exposes an OpenAI-compatible API and runs on Envoy, the open-source proxy S¹. For teams that want a managed path, Switchyard is available through OpenRouter using the model identifier nvidia/switchyard S¹.
Tanay Varshney, Chris Alexiuk, Ayush Agarwal, Annie Surla and Moon Chung described the system in a NVIDIA Technical Blog post on 11 August 2026 as a way to direct agent workloads to both specialised and frontier models, trading performance against cost P².
Two integrations have documented gaps that block some routing strategies
The LiteLLM integration is experimental and checkout-only, pinned to LiteLLM version 1.102.0 S¹. It supports Stage routing based on request history plus Random routing, but it cannot service the intermediate model calls that classifier or escalation algorithms require S¹. If your routing strategy needs a small model to classify a request before forwarding it to a larger one, the LiteLLM path will not handle that.
NeMo Relay versions 0.8.x and 0.9.0 can lose upstream error status codes and details when the Switchyard plugin is enabled, even for models outside its routes S¹. A failed call to a model Switchyard is not routing may still surface as a garbled error, which makes debugging harder.
The README also advises evaluating the complete agent and routing configuration against a single-model baseline S¹. This is sound guidance, and it matters because routing algorithms that rely on intermediate model calls add their own latency and cost, which can undercut the savings they are meant to deliver.
Coding-agent teams on single providers are the natural first adopters
A team running a coding agent like Claude Code against a single provider, paying frontier-model rates for every request including trivial ones, is the natural first user. Switchyard lets that team route simple requests to a cheaper open-weight model while reserving the frontier model for complex tasks, all behind the same API surface S¹P⁴. For a DevOps engineer managing that setup, the practical change is swapping one base URL in the agent config to point at Switchyard's OpenAI-compatible endpoint rather than maintaining separate clients for each provider S¹.
The one thing to check first: run your real workload through Switchyard against your current single-model baseline and compare total task completion cost, not per-call cost. The README itself warns that a cheaper model call does not guarantee a cheaper successfully completed task S¹, because a weaker model may fail and trigger retries that cost more than a single frontier-model call would have.
No one outside NVIDIA has independently verified that Switchyard reduces cost or improves latency on any workload; no error bars, no third-party benchmarks and no released security audit exist S¹. The repo is trending, but GitHub trending status does not imply production readiness. The NVIDIA-NeMo GitHub organisation, created 27 May 2025, lists 32 public repositories and 2,139 followers P³.
The next checkpoint for any team considering adoption is the NeMo Relay bug: if you run Relay 0.8.x or 0.9.0, test whether enabling the Switchyard plugin corrupts error reporting on models outside its routes before you ship it past a staging environment S¹.
Sources: S1 — NVIDIA-NeMo/Switchyard: Switchyard lets LLM applications route traffic · P2 — Route AI Agents Across Models with NVIDIA NeMo Switchyard | NVIDIA Tec · P3 — NVIDIA-NeMo · P4 — README.md · P5 — anthropics/anthropic-sdk-python
Written from 5 sourced items, 4 of them primary.