Advancements in Multimodal and Efficient Large Language Models
The landscape of open-weight artificial intelligence saw several technical developments this week, characterized by the release of a large-scale multimodal model, expanded local deployment support for Cohere architectures, and the introduction of a highly efficient fine-tuned model.
InclusionAI Releases Ming-3.0-audio-flash
InclusionAI has released Ming-3.0-audio-flash, a multimodal model designed for audio understanding and speech output. According to the model card hosted on Hugging Face, the model utilizes a Mixture-of-Experts (MoE) architecture, featuring 126 billion total parameters with 6.5 billion active parameters during inference.
The model is capable of processing audio alongside text instructions to perform tasks such as transcription, spoken Q&A, and reasoning regarding environmental sounds, music, and speech. It supports both English and Chinese, with claims of robustness against accents, dialects, and noisy recordings. For speech output, the model offers an optional “Talker” feature that utilizes built-in voices or reference audio.
The model is released under an MIT license. Deployment documentation includes paths for Transformers, SGLang, and vLLM-Omni. Reference configurations for high-performance setups include a vLLM-Omni profile requiring eight 80GB H800 GPUs to manage a 128K context window.
Cohere Expands llama.cpp Compatibility
Cohere has announced progress in making its model architectures compatible with llama.cpp, a tool widely used for local model execution. According to a statement from Cohere representative Nick, the team has merged support for the vision encoder of the North Micro Vision Instruct model. This development ensures that Cohere’s latest architectures are compatible with the llama.cpp ecosystem.
The North Micro Vision model serves as the foundation for several of the company’s recent releases. Additionally, Cohere Transcribe, a speech-to-text (STT) model, is now available for fine-tuning within Unsloth and Unsloth Studio. Independent reports from developers on community forums suggest high performance for the STT model on mobile hardware, with one user reporting speeds of 40x real-time on an iPhone 15 Pro Max and 90x real-time on an M3 MacBook Air.
Release of Tess-4.5-27B
A new fine-tuned model, Tess-4.5-27B, has been released by developer migtissera. Based on the Qwen architecture, the model is designed to optimize token efficiency without sacrificing intelligence.
According to data provided by the author, Tess-4.5-27B achieves comparable performance to its base model while using approximately 25% fewer tokens across 150 BenchLocal scenarios. Specific performance metrics cited include a 32% reduction in tokens for CLI-40 tasks and a 43% reduction for data extraction tasks. The developer notes that this reduction in token usage allows for faster response times and longer session durations before reaching context window limits. The model is currently available in GGUF format on Hugging Face.