NVIDIA Dynamo 1.6 lets its Rust endpoint picker scale out and strips Claude Code's billing header by default

NVIDIA’s Dynamo, the open-source serving layer that sits above vLLM, SGLang and TensorRT-LLM, shipped v1.6.0 on 22 October. The project’s README describes it as built in Rust for performance and Python for extensibility. The notes count 654 pull requests from 148 contributors, 58 of them outside NVIDIA.

Router and Kubernetes

The router gains a request-classifier hook that can defer a request or suggest a worker, and every request now flows through a policy queue ordered by deadline. Scoring and queue settings move into one --router-policy-config file, and the old flags are slated for removal in v1.7.0.

On Kubernetes, deployments that use the native Rust endpoint picker (EPP) can now run more than one replica and survive losing a pod. The legacy Go EPP still needs a single replica. The runtime now warns on sustained Tokio queue pressure and exports a count as dynamo_tokio_queue_overload_warnings_total.

Coding-agent traffic

Several changes target agent workloads:

1. Billing header stripped by default. Dynamo now removes the x-anthropic-billing-header preamble Claude Code adds, to save tokens and keep prompt caches stable. Set DYN_STRIP_ANTHROPIC_PREAMBLE=0 or pass --no-strip-anthropic-preamble to keep it.
2. Headers kept. An agent_headers map preserves the x-claude-code-* and x-codex-* families.
3. Sessions grouped. An opt-in session prefix index sends a subagent’s requests to the same worker as its parent session, so they share cache.
4. No guessing. Non-streaming /v1/responses calls no longer infer tool calls from plain assistant text.

Engines

Bundled engines move to SGLang v0.5.21, TensorRT-LLM v1.3.0rc29 and vLLM v0.31.0, so check your engine pins with the upgrade.

Why it matters

Coding agents send long, repetitive prompts with many subagents. Dynamo is tuning routing around that pattern, which is where cache hits and GPU cost are won or lost.