Technical Specifications and Architecture
OVHcloud has announced the development of “Olala mini 7A1B,” a new large language model utilizing a 7-billion-parameter architecture. According to technical details shared via Reddit, the model employs a Mixture-of-Experts (MoE) approach, featuring one billion active parameters distributed across 256 experts per layer.
The architecture incorporates a hybrid attention mechanism that combines Differential-TPA and Mamba-3, operating in BF16 format. This design allows only a fraction of the model to be utilized for each individual request, a feature intended to reduce memory requirements and increase processing speeds. Currently, the model supports a context window of 32,000 tokens.
Performance Metrics and Hardware Compatibility
Preliminary performance data suggests the model is optimized for efficiency across various hardware configurations. Reports indicate that the model can run on a GPU with as little as 8 GB of memory. In testing environments, it reportedly reached speeds of 160 tokens per second on an NVIDIA RTX A4500. Furthermore, the model is capable of running on a standard processor without a dedicated GPU, achieving speeds of approximately 50 tokens per second.
While these performance figures have been circulated on social media platforms, independent benchmarks have not yet been released to verify these specific throughput claims.
Release Timeline and Licensing
The official launch of Olala mini is scheduled to take place at the OVHcloud AI Summit on November 19. Once released, the model is expected to be available as open source on Hugging Face under the Apache 2.0 license.
According to unverified reports from community discussions, a version 1.1 is anticipated by the end of the year. This subsequent iteration is expected to extend the context window to 128,000 tokens and provide improvements in mathematical reasoning capabilities.
Market Reception and Observations
The announcement has met with a mixed response from the developer community. Some observers have noted that the current release lacks publicly available weights or comprehensive benchmarking data, leading to questions regarding the immediate utility of the announcement. Additionally, some commentators have expressed skepticism regarding whether the current iteration is a standalone base model or a fine-tuned version of an existing architecture.
As of the time of this report, the model weights have not been made available on the OVHcloud Hugging Face repository.
