Nvidia Open-Weights Nemotron 3.5 Super VL: Mamba-Transformer Hybrid Hits 1M Context

Nvidia released Nemotron 3.5 Super VL today — a 120B-parameter open-weight Mixture-of-Experts with 12B active parameters per token, a hybrid Mamba-Transformer backbone, and a 1M-token context window. Weights, datasets, and training recipes are on Hugging Face and OpenRouter under the OpenMDW license. Nvidia claims the model beats Claude Opus 5 and GPT-6.1 Sol on five specialized post-trained tasks. That is a vendor benchmark, so treat it as one.

The release is the second major open-weights drop in 24 hours after Meta's Muse Spark Open. Two frontier-scale open releases in one cycle is a market structure signal: the open-weights tier is now a multi-lab race, not a single-lab story.

The Architecture

The hybrid Mamba-Transformer backbone is the bet. Standard Transformers scale attention compute quadratically with sequence length, which is the bottleneck on long context. Mamba state-space model layers handle long-range dependencies with linear compute, which is what makes a 1M context window tractable. Nemotron 3.5 Super VL interleaves Mamba layers with standard self-attention layers, with the mix tuned at training time.

The result: a 120B MoE that holds a 1M context window with competitive retrieval performance. Independent long-context benchmarks (the LongBench variant, the Needle-in-a-Haystack test, the RULER benchmark) are not yet published for Nemotron 3.5 Super VL, but the architectural pattern is consistent with what Jamba, Zamba, and other hybrid-backbone models have shown: linear scaling on long sequences, Transformer-grade precision on short ones.

The 12B active parameter budget is conservative for a 120B MoE. By comparison, Muse Spark Open is 42B active on 816B total, and DeepSeek-V3 is 37B active on 671B total. Nemotron 3.5 Super VL is trading active compute for inference efficiency — at 12B active, the per-token cost is closer to a 12B dense model than a 120B MoE.

Why This Matters

The Mamba-Transformer hybrid is now a release architecture, not a research curiosity. Jamba (AI21), Zamba (Zyphra), and now Nemotron have all shipped hybrid models at scale. The pattern is established. For builders evaluating long-context models, the architectural choice is no longer "Transformer-only" or "Mamba-only" — it is "what Mamba/Transformer mix, tuned for which workload."

The 1M context tier is now the open-weights frontier. Claude Opus 5, Qwen 3 Max, Muse Spark Open, and now Nemotron 3.5 Super VL all hit 1M. Four independent labs, three different architectures, same ceiling. The 1M context tier is the new commodity for frontier-tier models. What separates them now is the reliability and the recall across the full context.

Nvidia is now a model lab, not just a chip vendor. With Nemotron 3.5 Super VL, Nvidia is shipping open-weight models that compete with frontier closed-weight releases on capability, with an open license that competes with frontier closed-weight releases on commercial usability. The vertical integration — chip + system + model — is the strategic play. The OpenMDW license is the hook to pull developers into the Nvidia stack.

Open-weights release cadence is now a multi-lab pattern. Meta, DeepSeek, Microsoft (via Phi and the Mistral partnership), Alibaba, and now Nvidia are all shipping open-weight models on a regular cadence. The release calendar is no longer "wait for Llama." Builders planning a model strategy should be evaluating on a 2–3 month cycle, not a 6–12 month cycle.

The Trade-Off

The "beats Claude Opus 5 and GPT-6.1 Sol on five specialized post-trained tasks" claim is Nvidia's. The methodology and the task selection are Nvidia's. Independent verification on broader benchmarks is not yet published. Treat the claim as a vendor number, not an audited result.

The 12B active budget also means Nemotron 3.5 Super VL is not directly competitive with Muse Spark Open or DeepSeek-V3 on tasks that benefit from a larger active compute per token (e.g., complex multi-step reasoning, long-form generation, code synthesis at scale). For pure long-context retrieval and document understanding, the architecture is well-suited. For general-purpose frontier work, the smaller active budget is a real constraint.

The Bottom Line

Nvidia released a 120B MoE with a 1M context window, a Mamba-Transformer hybrid backbone, and an open license. The architecture is the bet; the license is the lever. Independent benchmarks are pending, but the Mamba-Transformer hybrid pattern is now established at release scale. For long-context workloads, Nemotron 3.5 Super VL is on the short list. For general-purpose frontier work, wait for independent benchmarks before committing.