NVIDIA ReplaySSM Speeds Up Decoding in Hybrid Models Like Nemotron 3 and Qwen3.5

An NVIDIA technique that redesigns how SSM layer state is stored inside hybrid models, aiming to speed up decoding and reduce memory pressure during generation.

SSM layers use a fixed-size state that does not grow with context length, while attention layers rely on a KV cache that expands with text length, making SSM faster to decode but weaker at precise recall.

Hybrid SSM-based models such as NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash, using Mamba-2, Gated DeltaNet, or Kimi Delta Attention.

Yes. Lower inference latency and state memory reduce the cost of running hybrid models on local or cloud infrastructure, which matters for enterprises and governments deploying large models.