ResearchPapers, evidence & method
Nesformer runs Super Mario Bros. on a NES CPU compiled directly into Qwen3.5-4B transformer weights
Drew Breunig's open-source nesformer compiles a 6502 processor into a transformer with Qwen3.5-4B's architecture. It uses Gated DeltaNet state as RAM to run NROM games like Super Mario Bros. No weights were trained, and outputs can be checked exactly against reference traces.
From the Simon Willison article of October 30, 2026 and cited sources: A compiler calculated every weight, so none were learned. [2] [3] [4]
SOURCES[2] dbreunig/nesformer, GitHub github.com[3] Tracr: Compiled Transformers as a Laboratory for… arxiv.org[4] Qwenputer, Divyansh Agrawal divagr.com
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- Original source
- Simon Willison article, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- Studies
- arXiv 2301.05062 (Jan 2023) [3] date from arXiv ID
- When
- October 30, 2026 article published
- Covered
- Oct 31 edition
- Evidence
- 7 sources across 6 sites
- 1dbreunig/nesformer, Simon Willison's Weblogsimonwillison.net
- 2dbreunig/nesformer, GitHubgithub.com
- 3Tracr: Compiled Transformers as a Laboratory for…arxiv.org
- 4Qwenputer, Divyansh Agrawaldivagr.com
- 5Hacker News Digest, October 30, 2026, Zelizeli.app
- 6Nesformer mete un emulador de NES en Qwen3.5-4B…ecosistemastartup.com
Reach: not measured yet
A compiler calculated every weight, so none were learned. The checkpoint keeps Qwen3.5-4B's architecture but contains no original language weights. Only 9.3M of its 466.6M parameters are nonzero. This builds on Tracr's program-to-weights compilation and on Qwenputer, which added a compiled RISC-V CPU in front of a stock Qwen model. [2] [3] [4]
Each Gated DeltaNet head has a 128×128 state matrix, which nesformer uses as addressable memory cells. Each CPU instruction becomes a short token record, averaging 1.1–1.3 tokens per instruction on real game code, compared with several in Qwenputer. It runs on one CPU core but is 150–290× slower than the original console. Graphics, sound and controllers are handled by host software, not by the model. [2] [4] [6]
The author reports exact matches with the nestest reference log over 3,348 instructions and agreement with the py65 emulator over 5.7M instructions. Free-running test cartridges match reference traces for more than 700,000 tokens. No independent re-run appears in the reviewed sources. [2]
Breunig says Claude Opus 5.5 built the project from two casual prompts. According to Simon Willison's account, the model found and used a 15-year-old NES CPU conformance suite, which suggests existing reference tests are what make such fast AI-assisted builds checkable. [1]
exact trace matches show a transformer can run a CPU deterministically and verifiably. Implication: compiled modules could provide auditable, exact computation inside models.
Read the full assessment
Nesformer has no language weights, though, so combining such a module with a working LLM remains untested here.
Executive brief
Drew Breunig's nesformer runs Super Mario Bros. on a Nintendo (NES) CPU inside a transformer, and no one trained it. A compiler calculated every weight. Agrawal's earlier qwenputer had to be added on top of a working language model. Nesformer's checkpoint has no language weights at all. It keeps Qwen3.5's architecture and uses the model's own memory mechanism as the console's RAM. It is 150–290× slower than real hardware, and Breunig says Claude Opus 5.5 wrote it from two short prompts (Simon Willison). The project shows that a transformer can act as a fully deterministic, testable machine. It is not a product.
What changed and event timeline
Tracr compiles programs into transformers
DeepMind researchers built a compiler that turns readable programs into decoder-only transformer weights. It was meant as a lab for checking interpretability methods ().
Qwenputer puts a RISC-V CPU in Qwen 3.5 4B
Divyansh Agrawal added 11 compiled layers in front of the stock 32-layer model. Gated DeltaNet state served as exact RAM, and the CPU passed the official RISC-V tests ().
Nesformer reaches the Hacker News front page
The post was titled "A NES console runs inside Qwen weights that nobody trained." Upvote counts differ between snapshots: 175 in one listing and 216 in another (;).
Willison publishes the build prompts
Breunig shared on HN the two Opus 5.5 prompts behind the project, ending with "I think we should build it! According to Willison, Opus found and used a 15-year-old set of NES CPU conformance tests ().
Capabilities and access
- Format: Qwen3.5-4B architecture. 466.6M parameters in bf16 (933 MB), of which only 9.3M are nonzero, so the file compresses to 3.3 MB (repo).
- Inside the weights: a 6502 core with all 151 official opcodes, every addressing mode, flags, stack, BRK/NMI, 2 KB of work RAM and 32 KB of program ROM.
- Outside the weights: the host program handles the PPU (picture processor: scanline renderer and palette), controllers and APU (sound) registers.
Read the full section
- Format: Qwen3.5-4B architecture. 466.6M parameters in bf16 (933 MB), of which only 9.3M are nonzero, so the file compresses to 3.3 MB (repo).
- Inside the weights: a 6502 core with all 151 official opcodes, every addressing mode, flags, stack, BRK/NMI, 2 KB of work RAM and 32 KB of program ROM.
- Outside the weights: the host program handles the PPU (picture processor: scanline renderer and palette), controllers and APU (sound) registers.
- Runs on: one CPU core, no GPU. About 1,650 tokens/s with numba, about 210 with numpy, about 10 with stock
transformers. - Games: NROM (mapper 0) cartridges only.
Technical analysis for researchers and developers
- Memory: Each Gated DeltaNet head keeps a 128×128 state matrix, and nesformer treats it as addressable memory cells.
- Execution: Each CPU instruction is written out as a short token record, and the record's last token commits it.
- Determinism: The weights are computed, not learned, so the output can be checked exactly against reference logs. The code is public.
Read the full section
- Memory: Each Gated DeltaNet head keeps a 128×128 state matrix, and nesformer treats it as addressable memory cells. Qwenputer showed the trick: set the gate and write strength to 1 and use one-hot keys, and each write replaces exactly one row (Qwenputer; repo).
- Execution: Each CPU instruction is written out as a short token record, and the record's last token commits it. Real game code averages 1.1–1.3 tokens per instruction, against several tokens per instruction in qwenputer.
- Determinism: The weights are computed, not learned, so the output can be checked exactly against reference logs. The code is public.
Claims and evidence
- It matches nestest's reference log for 3,348 instructions and agrees with the py65 emulator over 5.7M instructions (repo).
- Free-running test cartridges match reference traces across more than 700,000 tokens (same source).
- Speed is 150–290× slower than the console, at 1–6 seconds per frame (ecosistemastartup). An idle-loop shortcut raises throughput 2.6–4.2× (repo).
Read the full section
- It matches nestest's reference log for 3,348 instructions and agrees with the py65 emulator over 5.7M instructions (repo).
- Free-running test cartridges match reference traces across more than 700,000 tokens (same source).
- Speed is 150–290× slower than the console, at 1–6 seconds per frame (ecosistemastartup). An idle-loop shortcut raises throughput 2.6–4.2× (repo).
- All of these test results come from the author. The reviewed sources contain no independent re-run.
- The claim that Opus 5.5 built it rests on Breunig's HN comment as quoted by Willison. The repo itself does not mention AI assistance.
Context and prior work
- Tracr (NeurIPS 2023 spotlight) showed that weights compiled from programs give researchers models whose internals are known in advance.
- Qwenputer scaled the idea up to a whole CPU and verified it: zero mismatches across 230,000 output positions and 18M free-running instructions.
- Nesformer's tests come from the community NES test-ROM collection, which includes Blargg's CPU and PPU tests (nes-test-roms).
Read the full section
- Tracr (NeurIPS 2023 spotlight) showed that weights compiled from programs give researchers models whose internals are known in advance.
- Qwenputer scaled the idea up to a whole CPU and verified it: zero mismatches across 230,000 output positions and 18M free-running instructions.
- Nesformer's tests come from the community NES test-ROM collection, which includes Blargg's CPU and PPU tests (nes-test-roms).
- Willison's prompt describes this as like rebuilding the NES on FPGAs (reprogrammable chips), as MiSTer hobby boards do (Simon Willison).
Limitations, safety and contested findings
- Unsupported: bank-switching cartridges, unofficial opcodes, running code from RAM, and audio/APU IRQs. Sprite collision is approximate (repo).
- The model does not render graphics. The PPU runs in ordinary host software, so this is not a whole console inside the model.
- "Inside Qwen" means two different things. Willison describes the emulator as baked "into the weights of Qwen3.5-4B" (Simon Willison).
Read the full section
- Unsupported: bank-switching cartridges, unofficial opcodes, running code from RAM, and audio/APU IRQs. Sprite collision is approximate (repo).
- The model does not render graphics. The PPU runs in ordinary host software, so this is not a whole console inside the model.
- "Inside Qwen" means two different things. Willison describes the emulator as baked "into the weights of Qwen3.5-4B" (Simon Willison). The repo says the checkpoint is entirely compiled, with no original Qwen weights (repo). Qwenputer, by contrast, did keep the stock weights (Qwenputer).
Business and practitioner implications
- No commercial use case. At 150–290× slower than 1985 hardware, this does nothing a normal emulator can't.
- The more useful lesson is about AI-assisted building. Existing reference test suites appear to be what makes such fast builds checkable.
- Exact, auditable computation: In Qwenputer the compiled CPU sits in the same network as the language model, which suggests a route to exact computation inside a model without external tool calls.
Read the full section
- No commercial use case. At 150–290× slower than 1985 hardware, this does nothing a normal emulator can't.
- The more useful lesson is about AI-assisted building. Breunig reports that a frontier coding model, given two casual prompts, found an old conformance suite and built a working system (Simon Willison). Existing reference test suites appear to be what makes such fast builds checkable.
- Exact, auditable computation: In Qwenputer the compiled CPU sits in the same network as the language model, which suggests a route to exact computation inside a model without external tool calls. Nesformer drops the language weights, so it doesn't test that combination.