MIT CSAIL researchers published on November 2, 2026 an experiment that sounds like science fiction: OpenAI's GPT-6 Astra can directly read and write the weights of a neural network, with no gradient descent and no training loop. The work, titled “Sparks of Weight-Space Programming”, is authored by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab — and the authors themselves describe it as very early results, with no peer review or independent replication yet.
The core idea is what the authors call Weight-Space Programming: instead of training a model to adjust its weights, ask a frontier model, in natural language, to interpret another network's raw numbers and rewrite them toward a goal. In the smaller experiments, Astra identified from weights alone which game a network had been trained to play — Breakout, at 99% confidence, from 17,253 weights — fixed a driving policy stuck on a ramp, reaching 100% success across 1,024 episodes after editing 3.5% of the weights, and wrote game networks from scratch: in Breakout, its one-shot network scored 517.4, against 15.8 for a PPO model trained for 100 million steps.
The Qwen3.5-0.8B case: 0.016% of the weights, six times the math
The most discussed experiment used Qwen3.5-0.8B, Alibaba's open model with 800 million parameters. Given a Python notebook over the weights and a budget of forward passes, Astra traced a known defect of the model — reasoning loops, which Qwen's own model card warns about — to a single direction written by the MLPs of layers 15 and 19, and subtracted that direction from 64 output columns in each layer: 131,072 weights changed, or 0.016% of the total. With 50 forward passes, MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1% and IFEval, which measures instruction following, from 21.0% to 62.3%. The share of MATH-500 answers hitting the 32,768-token limit fell from 93% to 33.1%.
The authors themselves apply the brakes: the fix removed an obstacle hiding an ability, it did not create a new ability. The control experiment proves it: on Gemma 3 270M, a model that rarely loops, no edit moved MATH-500 by more than about one point. Likewise, Astra could not tell from weights alone which of two models had been trained to be sycophantic — it only succeeded when allowed to inspect attention patterns. The authors published the complete logs of every experiment and are now building a benchmark to measure this capability rigorously.
Image credit: Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, in the post “Sparks of Weight-Space Programming” (image from the Breakout experiment).
Sources: Rikiya Takehi e coautores (MIT CSAIL) — Sparks of Weight-Space Programming; ExplainX — GPT-6 Astra Edits Qwen Weights.
Transparency: this article was produced with the assistance of artificial intelligence, cross-checking public posts on X and internet sources. Consult the original sources to confirm the details.
Muse
