ResearchPapers, evidence & method
Qwen 3.8 27B on a laptop identifies which games small RL networks were trained on by reading their raw weights
A quantized 27B Qwen model running on a laptop, with no tools, identified three of four games that small reinforcement-learning networks were trained on by reading their raw weights, suggesting MIT's weight-reading task does not require a frontier model.
From the Simon Willison article of November 3, 2026 and cited sources: Simon Willison ran MIT's game-identification prompts on a 17GB Q4_K_M build of Qwen 3.8 27B on his laptop, with no tools. [1]
SOURCES[1] Qwen 3.8 27B on my laptop can interpret model weights, too… simonwillison.net
AT Pulse research deskPublished · Updated · 4 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Simon Willison ran MIT's game-identification prompts on a 17GB Q4_K_M build of Qwen 3.8 27B on his laptop, with no tools. [1]
- Original source
- Simon Willison article, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- November 3, 2026 article published
- Covered
- Nov 4 edition
- Evidence
- 8 sources across 7 sites
- 1Qwen 3.8 27B on my laptop can interpret model weights, too…simonwillison.net
- 2There's a buzzy new paper about GPT-6 Astra being able to…x.com
- 3Sparks of Weight-Space Programming (MIT CSAIL)rikiyat.github.io
- 4VentureBeat coverageventurebeat.com
- 5Honnibal: Astra read/write is misleadinghonnibal.dev
- 6Goedecke: LLMs are not superhuman at reading and writing…seangoedecke.com
StrongHow it’s measured
Hacker News · 665 likes · 193 comments
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
Simon Willison ran MIT's game-identification prompts on a 17GB Q4_K_M build of Qwen 3.8 27B on his laptop, with no tools. It correctly named Pac-Man, Pong and Breakout and missed Freeway. Each answer took 46 to 65 minutes of reasoning, and each game was run only once. [1]
The task comes from MIT CSAIL's Weight-Space Programming paper. It says GPT-6 Astra can identify, write and repair network weights without gradient descent. One example is an edit to Qwen3.5-0.8B that raised MATH-500 from 7.9% to 48.1%. Those figures come from the authors. An unaffiliated reproduction tested only 18 problems and did not re-measure MATH-500. [3] [4] [7]
Critics dispute the framing. Every experiment on a real language model used Python and model execution. The game networks are small, with labeled inputs that map onto grid cells, so simple norms of the first-layer weights show which parts of the board each network watches. The math fix works like standard steering by removing a looping direction. The authors themselves say their edits remove obstacles but do not add new abilities. [5] [6] [3]
The evidence is a single local run per game, on networks of about 17,000 weights.
Read the full assessment
The implication: weight-level auditing and patching may be within reach of open models on consumer hardware, though slowly and so far only for tiny networks or known failures.
Executive brief
A 17GB quantized open model on a laptop, Qwen 3.8 27B, worked out which game three small reinforcement-learning networks had been trained to play by reading nothing but their raw weights as text. It had no tools, and each answer took 46 to 65 minutes of reasoning. The task comes from MIT CSAIL's "Sparks of Weight-Space Programming", which reported that GPT-6 Astra could read, write and repair network weights. Simon Willison's test suggests that ability isn't limited to frontier models. Critics say the paper's headline results lean on ordinary ML tooling, and Willison himself says a patient human with pencil and paper could probably do the same.
What changed and event timeline
MIT publishes Weight-Space Programming
Takehi, Teoh, Zhang and Khattab say GPT-6 Astra can identify, write and fix network weights without gradient descent. They released the logs and code alongside it (;).
Press amplifies the Qwen math result
VentureBeat headlined the 7.9%→48.1% MATH-500 gain and quoted both optimistic and alarmed commentators ().
Honnibal calls the framing misleading
He argues that every experiment on a real language model used Python and model execution, not raw weight-reading ().
Goedecke disputes "superhuman."
He argues that a competent ML researcher could do each task, and that the Qwen fix follows standard steering methodology ().
Local model repeats the no-tools test
Willison ran the paper's game-identification prompts on Qwen 3.8 27B. It got Pac-Man, Pong and Breakout right and missed Freeway ().
Capabilities and access
- GPT-6 Astra: used in every experiment in the paper.
- Qwen 3.8 27B: run as a Q4_K_M GGUF (Hugging Face) in LM Studio through the
llm-lmstudioplugin. - Kimi K3.1 Pro: reached through OpenRouter. It identified Breakout in 9m41s using 19,493 reasoning tokens (gist).
Read the full section
- GPT-6 Astra: used in every experiment in the paper.
- Qwen 3.8 27B: run as a Q4_K_M GGUF (Hugging Face) in LM Studio through the
llm-lmstudioplugin. It needed a context window longer than the 8,192-token default and a raised timeout (Willison). - Kimi K3.1 Pro: reached through OpenRouter. It identified Breakout in 9m41s using 19,493 reasoning tokens (gist).
- Prompts: self-contained, and identical except for the weights (exp1_which_game).
Technical analysis for researchers and developers
- Game-ID setup: each network is the same MLP (500→32→32→5, 17,253 weights), trained with PPO on a 10×10 grid version of one of four games.
- Why it's tractable: the input channels map directly to grid cells and carry labels such as "agent."
- Qwen3.5-0.8B edit: the model had Python kernel access and could run 50 forward passes on practice problems.
Read the full section
- Game-ID setup: each network is the same MLP (500→32→32→5, 17,253 weights), trained with PPO on a 10×10 grid version of one of four games. The model is not shown a list of answer options (paper).
- Why it's tractable: the input channels map directly to grid cells and carry labels such as "agent." Goedecke notes that taking L2 norms of the first-layer weights shows which regions of the board the network watches (Goedecke). Qwen's reasoning traces do roughly this by hand, indexing rows and channels (Willison).
- Qwen3.5-0.8B edit: the model had Python kernel access and could run 50 forward passes on practice problems. It edited 131,072 parameters, about 0.016% of the model. Sampling settings were temperature 0.6, top-p 0.95, top-k 20 and presence penalty 1.5 (paper).
- Reproducibility: logs, code and the edited model are public.
Claims and evidence
- GPT-6 Astra identified all four games (paper). This is reported by the authors only.
- A one-shot Breakout network scored 517.4 against PPO's 25.3 (paper). This is also author-reported and has no outside check.
- MATH-500 rose from 7.9% to 48.1% and IFEval from 21.0% to 62.3% (paper). It did not re-measure MATH-500 (wsp-reproduction).
Read the full section
- GPT-6 Astra identified all four games (paper). This is reported by the authors only.
- A one-shot Breakout network scored 517.4 against PPO's 25.3 (paper). This is also author-reported and has no outside check.
- MATH-500 rose from 7.9% to 48.1% and IFEval from 21.0% to 62.3% (paper). An unaffiliated GitHub reproduction ran on Kaggle T4 GPUs and tested only 18 problems. The base model got 2 right, the reproducers' own edit got 15, and the paper's edit got 17. It did not re-measure MATH-500 (wsp-reproduction).
- Qwen 3.8 27B got 3 of 4 games, with self-stated confidence of 72–88% (Willison). This is a single run per game, not a benchmark.
Context and prior work
- The Qwen fix removes a "looping" direction. That works like known steering and ablation methods, which is Goedecke's point (Goedecke).
- The loop failure it targets is one that Qwen's own model card already warns about (Willison).
- Other people have started building on the idea, for example an agent-driven weight-writing repo (csinva).
Limitations, safety and contested findings
- Scope: the no-tools results cover only tiny networks of 17K–172K parameters. Every experiment on a real language model used code (Honnibal).
- Repair, not new ability: the authors say edits remove obstacles but don't add missing capabilities.
- Weak spots in the local test: Qwen missed Freeway, and its reasoning traces are close to unreadable (Willison).
Read the full section
- Scope: the no-tools results cover only tiny networks of 17K–172K parameters. Every experiment on a real language model used code (Honnibal).
- Repair, not new ability: the authors say edits remove obstacles but don't add missing capabilities. Edits to Gemma 3 270M, which rarely loops, gave negligible gains (paper).
- Weak spots in the local test: Qwen missed Freeway, and its reasoning traces are close to unreadable (Willison).
- Safety concern: Pause AI's CEO warned the technique could speed up AI progress dangerously (VentureBeat).
Business and practitioner implications
- Small models: targeted weight patches look most promising for small models with a known, diagnosable failure.
- Local auditing: reading weights without tools runs on consumer hardware, but slowly, at about an hour per small network (Willison).
- Hype: Willison rejects claims that source code is obsolete.
Read the full section
- Small models: targeted weight patches look most promising for small models with a known, diagnosable failure. Teams would still need regression tests, version control and a cost comparison against retraining, and VentureBeat notes there are no deployment examples yet (VentureBeat).
- Local auditing: reading weights without tools runs on consumer hardware, but slowly, at about an hour per small network (Willison).
- Hype: Willison rejects claims that source code is obsolete.
The source trail.
Sources (8)
Qwen 3.8 27B on my laptop can interpret model weights, too (Simon Willison)
simonwillison.netThere's a buzzy new paper about GPT-6 Astra being able to directly interpret model weights (prompts full of floating point numbers) - I tried it with Qwen 3.8 27B on my l
Related coverage; assess separately
x.com