Expansion of NVIDIA Nemotron Model Family

NVIDIA CORP has expanded its suite of artificial intelligence models with the release of the Nemotron-3.5 Super VL-120B-A12B model family. According to technical documentation and community discussions on platforms such as Reddit, the release includes several versions, including base, BF16, and NVFP4 formats, as well as implementations utilizing LiLiCorr speculative decoding.

The 120B-class model is designed with a hybrid architecture that combines Mamba-2 and Transformer layers. While NVIDIA has not released a formal technical paper detailing the specific attention mechanism used in the Transformer layers, the inclusion of Mamba-2 components suggests the model is engineered to maintain inference speed and competence even as context length increases. This architecture is reportedly intended to benefit Retrieval-Augmented Generation (RAG) tasks by allowing for large amounts of retrieved content to be processed efficiently.

Model Variants and Optimization

The release includes a specialized variant known as “Nemotron-3.5 Bolt-74B-A9B,” which is described by users as a heavily compressed version of the Super VL model. Rather than a standard fine-tune, the Bolt version appears to be a product of pruning and quantization-aware training (QAT), designed to retain much of the original model’s capability while significantly reducing its size for more accessible hardware.

The model card indicates that the Nemotron-3.5 series is oriented toward code generation and agentic tasks. While some community members have noted that its performance in specific benchmarks may vary compared to competitors like Qwen, the model offers a 1-million-token context window.

The development of the Nemotron series highlights NVIDIA’s continued investment in linear layers. Discussions among AI researchers on Reddit suggest that NVIDIA is among the few American companies actively developing these architectures. There is ongoing speculation regarding the potential for a “Nemotron 4” to incorporate advanced features such as Gated Delta Net 2 or specific methods for managing residuals, such as mHC or Gated Residuals.

While the Nemotron-3 Nano-30B-A3B model from the previous year remains a point of reference for users, the current 3.5 series represents a significant increase in scale and complexity. As the models become available on platforms like Hugging Face, the developer community is expected to focus on fine-tuning the models for specific applications, such as creative writing or specialized coding tasks. Currently, various quantized versions (GGUF) are being developed by third parties like Unsloth and ggml-org to facilitate broader hardware compatibility.