A chatbot can write an essay about a word and still fail to spell it backwards. The reason, classified here as CONFIRMED by the original paper, is that most language models never see letters: before any reading happens, a program called a tokenizer cuts text into chunks of words or word fragments, and the model learns from those chunks. Researchers at the Allen Institute for AI (Ai2) and the University of Cambridge have found a shortcut to fix this without training a model from scratch.
The method, called byteification, was published in Nature on 7 October 2026, in a paper led by Benjamin Minixhofer with Valentin Hofmann as senior author. Instead of rebuilding the model, the team kept the already trained core and swapped the input and output: new, smaller parts learn to read bytes — the basic units computers use to represent text — and group them into chunks the core understands. The process used about 49.1 billion tokens, less than 1% of the training normally needed to build an equivalent model from scratch.
Four open models of up to about 8 billion parameters were converted: Olmo 3 7B became Bolmo 7B, OLMo 2 1B became Bolmo 1B, Qwen3 8B became Bwen 8B and Llama 3 8B became Blama 8B. On general tests of science, maths, coding, reading and general knowledge, the converted models stayed close to the originals: Bolmo scored 62.5 versus 62.3 for Olmo 3, and Bwen scored 69.8 versus 70.3 for Qwen3. On letter tasks — spelling, reversing, swapping or deleting characters — the jump was large: Bolmo scored 78.1 versus 54.3 for Olmo 3, and Blama scored 81.0 versus 57.2 for Llama 3. On science, technology, engineering and maths questions, Bolmo beat an earlier byte-reading model trained from scratch by 16.5 points.
There are clear limits, and the paper itself points them out. Part of the gain came from extra training: about 75 million tokens of letter exercises, all in English. An unconverted Olmo 3 trained on the same mix already reached 70.7 on letter tasks — so byte reading added the rest, but does not explain everything on its own. The converted models were also slightly worse at writing working code on the first try, although they succeed more often when allowed many attempts. And none of this has been tested on much larger models or in languages other than English.
The practical value is cost: converting a finished model costs a fraction of full training, and the models, code and data were released so other groups can test the limits. For code, biological sequences and languages whose scripts are poorly covered by fixed token vocabularies, reading raw text may be the difference between a model that guesses letters and one that actually sees them.
Fontes
- Cobertura: Earth.com — Scientists give AI chatbots a new way to read, with surprising results
- Cobertura: AI Weekly — Byteification turns Olmo, Llama, Qwen into byte-level LLMs
- Cobertura: Allatra — A Language Model Rebuilt to Read Every Letter of Every Word
Image credit: Simple neural network diagram by Wiso, via Wikimedia Commons, in the public domain. Illustrative image; it does not show the models described in the article.
Transparency: this article was prepared with the help of artificial intelligence, based on cross-checking the public internet sources listed above. Consult the original sources to confirm the details before making any decision.
— Muse
