Cerebras puts disaggregated inference into production with AMD GPUs and CS-3 systems

Cerebras has put disaggregated inference into production: AMD Instinct GPUs process the prompt (the prefill stage) and the company's CS-3 systems generate the responses (the decode stage). According to the announcement published by the company on Tuesday (17) on its official blog, the combination delivers 5 times more tokens per megawatt than a CS-3-only configuration, with no loss in token generation speed.

The result was measured in production on the Cerebras Inference Cloud, on a shared GPT-OSS-120B endpoint on November 14, comparing throughput per megawatt before and after enabling disaggregation. It is the company's own measurement, on a single endpoint, with no independent measurement published — and the announcement gives no absolute tokens-per-second or power figures.

Why split prefill and decode

An inference request has two main stages. In prefill, the model processes the prompt and produces intermediate state; in decode, it uses that state to generate output tokens, one at a time. Because decode is autoregressive — each step re-reads the model's data from memory — memory bandwidth matters a lot. That is where Cerebras bets on its advantage: its wafer-scale chips pack tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth on a single chip — 350 times the SRAM capacity and 5,400 times the memory bandwidth of a typical GB300 GPU, according to the company.

By shifting prefill to the GPUs, Cerebras systems devote all of their serving capacity to generating responses. The company had already mentioned the launch on its third-quarter earnings call on November 12 — covered by Geekiki in an article about Cerebras' earnings — and Tuesday's announcement adds the production measurement and the package for infrastructure partners.

The partner package and what's next

The solution was also built for partners to adopt: they can deploy disaggregated inference clusters from a Cerebras reference architecture, co-locating their prefill accelerators and Cerebras systems on the same network. For software integration there are two options: a full stack — with a framework to route requests between prefill and decode pools, a prefill engine compatible with vLLM or SGLang, the Cerebras decode engine and the KV Portal software, which transfers intermediate state between the engines over RDMA — or a decode-only SDK, for those who prefer to bring their own framework and prefill engine.

The first early-access partners are Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute, and Cerebras says it is expanding access to other partners. The company plans to deploy disaggregated inference across hundreds of megawatts in its inference cloud and in partner deployments by the end of 2027 — alongside the recently announced CS-4 and 700 MW of planned data center capacity. In the same month, the company also launched Tungsten, its high-level language for programming the wafer-scale chip, covered by Geekiki in an article about Tungsten.

Image: Cerebras Systems logo, via Wikimedia Commons (public domain).

Sources

Transparency note: this article was produced with the help of artificial intelligence, based on cross-checking public posts on X and sources on the internet, and reviewed against the original sources cited above. We recommend consulting the original sources to confirm the data before making any decision.

Por Muse