OVHcloud launches Olala mini 7A1B, its first open-source LLM with 1 billion active parameters

OVHcloud announced on Tuesday (17) Olala mini 7A1B, the first model in its new LLM family, Olala. The compact model was trained from scratch by the French company and will be released as open source on Hugging Face, under the Apache 2.0 licence. The official launch takes place at the OVHcloud AI Summit on 19 November.

Olala mini has a 7-billion-parameter architecture, but uses only 1 billion active parameters per request: the active parameters are spread across 256 experts per layer, in the design known as mixture of experts. Its hybrid attention combines Differential-TPA and Mamba-3, in BF16 format — and because only a fraction of the model is used for each request, it runs faster and uses less memory.

In practice, the model runs on a GPU with as little as 8 GB of memory, reaching 160 tokens per second on an RTX A4500 — and even without a GPU, on a standard processor, at 50 tokens per second. Its current context window is 32,000 tokens.

Trained on European public supercomputers

Olala mini was trained on three European public supercomputers — Jupiter, MeluXina and Jean Zay — equipped with GH200, A100 and H100 GPUs. According to OVHcloud, the public support is equivalent to €7 million in computing hours and was decisive for the project, which also gave rise to the company's AI lab, OVHai LLM.

The release package includes the model weights, the pre-trained base model, the technical report and compliance documentation for the AI Act, the European Union's artificial intelligence legislation.

The model already has a first production user: French search engine Qwant has adopted Olala mini for its flash answer feature, with answers 20% faster and running costs divided by three compared with the model previously used, according to OVHcloud.

What comes next

By the end of the year, version 1.1 is expected to extend the context window to 128,000 tokens and improve performance in mathematical reasoning. Version 2.0, with more than 100 billion parameters, will bring models specialised in three areas: agentic AI for businesses, cloud DevOps and cybersecurity.

For OVHcloud CEO Octave Klaba, the decision to train its own models is strategic: the company does not want to depend only on the generosity of other labs to share LLMs as open source. According to him, training costs have fallen 20-fold in three years, and it is now possible to build LLMs profitably.

OVHcloud is Europe's leading cloud provider, with 46 data centres and 1.7 million customers in more than 140 countries. The bet on an in-house, open model trained on European public infrastructure reinforces the company's technological sovereignty message — the same theme as the sovereign AI partnership between HCLTech and Cohere announced today, which we also covered on Geekiki.

Image credit: OVHcloud headquarters in Roubaix, France; photo by Velvet, CC BY-SA 4.0, via Wikimedia Commons.

Sources

This article was produced with the assistance of artificial intelligence, based on cross-referencing public posts and internet sources. Consult the original sources to confirm the information.

Por Muse