ResearchPapers, evidence & method
Zoom In co-author Nick Cammarata says the cat-and-car neuron was misread and argues most neurons are already interpretable
In a long Goodfire essay, Nick Cammarata argues that apparent polysemanticity comes mainly from older architectures and weak tools, not fundamental superposition. He cites a preliminary Qwen3-32B audit. Prominent interpretability researchers call the claim overstated, and no technical paper has appeared.
From the MIT Technology Review · AI article of October 19, 2026 and cited sources: Cammarata reinterprets the canonical cat-and-car neuron from the 2020 Zoom In work as a detector for a shared V-shape. [1] [2] [3]
SOURCES[1] MIT Technology Review: "We were looking at it wrong" technologyreview.com[2] Goodfire: Neural Networks Are Already Clean already-clean.goodfire.com[3] AI Weekly on Zoom In aiweekly.co
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Cammarata reinterprets the canonical cat-and-car neuron from the 2020 Zoom In work as a detector for a shared V-shape. [1] [2] [3]
- Original source
- MIT Technology Review · AI article, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 19, 2026 article published
- Covered
- Oct 20 edition
- Evidence
- 8 sources across 8 sites
- 1MIT Technology Review: "We were looking at it wrong"technologyreview.com
- 2Goodfire: Neural Networks Are Already Cleanalready-clean.goodfire.com
- 3AI Weekly on Zoom Inaiweekly.co
- 4Toy Models of Superpositionastro.paperswithcode.com
- 5Trilogy AI: Goodfire says neural networks are already cleantrilogyai.substack.com
- 6LessWrong linkpost and discussionlesswrong.com
QuietHow it’s measured
Hacker News · 2 likes · 1 comment
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
Cammarata reinterprets the canonical cat-and-car neuron from the 2020 Zoom In work as a detector for a shared V-shape. He points to pool-table corners and bullet-train noses that activate it and an O-shape that switches it off. He proposes 'recursive generalization', where a neuron branches into related subconcepts instead of packing unrelated features together. [1] [2] [3]
The essay credits residual-era architectures for much cleaner neurons. It reports 99–100% easily interpretable neurons in ConvNeXt-medium's last layer, compared with about 5% in InceptionV1. In a self-reported, preliminary audit of Qwen3-32B, it finds that about 90% of neuron poles have one main job, about 7% have two related jobs and 2–4% remain unexplained. [2]
Coverage of the work describes two validation checks. Removing eight Santa-related neurons cut a near-certain next-token probability to 6%. Automated neuron descriptions predicted activations in new contexts 76% of the time, compared with 11% for near-miss descriptions. [5]
Leading researchers dispute the significance. Chris Potts calls it little more than a hunch that lacks rigorous evaluation. Jacob Hilton says real understanding requires circuits, not single neurons. Neel Nanda sees no major change to the big picture, noting that sparse autoencoders find structure that neurons miss. [1] [6]
If most neurons have one main job, teams could audit open-weight models by labelling neurons directly rather than first building sparse-autoencoder pipelines.
Read the full assessment
The evidence so far covers single neurons, not circuits or end-to-end behavior, so monitoring gains remain an implication.
Executive brief
Nick Cammarata co-authored the 2020 work that made the "cat-and-car" neuron the standard example of tangled, polysemantic neurons. He now says the field misread it. In a 26,000-word essay at Goodfire, he argues that the neuron responds to a shared V-shape, and that his preliminary audit of Qwen3-32B finds about 90% of neuron "poles" have one main job (Neural Networks Are Already Clean). If that holds, many neurons could be studied directly, which would make models easier to monitor. Leading interpretability researchers, including Neel Nanda, Jacob Hilton and Chris Potts, say the claim is overhyped. No technical paper has been published yet (MIT Technology Review).
What changed and event timeline
Zoom In makes polysemanticity canonical
Olah, Cammarata and co-authors used InceptionV1 to argue that networks form readable circuits. The paper also highlighted polysemantic neurons that respond to both cat faces and car fronts ().
Superposition formalized
Anthropic's toy models showed that networks can store extra sparse features in superposition. Later work focused on untangling neurons, often with sparse autoencoders (SAEs) ().
Cammarata publishes the essay
He argues that apparent polysemanticity mostly comes from architecture problems and weak tools, not from fundamental superposition (;).
- Mid Oct 2026
Experts push back
On LessWrong, Nanda doubts this is progress toward full reverse-engineering. Other commenters warn that it overclaims and could repeat earlier SAE hype ().
Mainstream coverage
MIT Technology Review reports celebration alongside skepticism from Potts, Hilton and Nanda. On Hacker News the essay reached 273 points, with reactions split (;).
Capabilities and access
- What it is: an essay with interactive "microscope" pages for browsing neurons. No code repository or dataset is listed (Goodfire essay).
- Models studied: InceptionV1, small/medium/large ConvNeXt, DINOv2, vision transformers and Qwen3-32B (Goodfire essay). Some coverage also mentions trillion-parameter Kimi K3 (daily.dev).
- Contributors: Curt Tigges and Tarun Gogineni.
Read the full section
- What it is: an essay with interactive "microscope" pages for browsing neurons. No code repository or dataset is listed (Goodfire essay).
- Models studied: InceptionV1, small/medium/large ConvNeXt, DINOv2, vision transformers and Qwen3-32B (Goodfire essay). Some coverage also mentions trillion-parameter Kimi K3 (daily.dev).
- Contributors: Curt Tigges and Tarun Gogineni.
- Still to come: a companion technical paper on language-model neurons.
Technical analysis for researchers and developers
- Proposed reframing: "recursive generalization." On this view, a neuron branches into related subconcepts rather than packing unrelated features together (Goodfire essay).
- Architecture matters: the essay reports 99–100% easily interpretable neurons in ConvNeXt-medium's last layer, versus about 5% in the original InceptionV1.
- The essay says the "one main job" counts depend on threshold choices and on how weak activations are treated.
Read the full section
- Proposed reframing: "recursive generalization." On this view, a neuron branches into related subconcepts rather than packing unrelated features together (Goodfire essay).
- Architecture matters: the essay reports 99–100% easily interpretable neurons in ConvNeXt-medium's last layer, versus about 5% in the original InceptionV1. It credits residual-era architectures for the difference.
- Method: agent-assisted labelling. The essay says the "one main job" counts depend on threshold choices and on how weak activations are treated.
- Validation reported in coverage: removing eight Santa-related neurons cut the next-token probability from near-certain to 6%. Automated neuron descriptions predicted activations in new contexts 76% of the time, versus 11% for near-misses (Trilogy AI).
- Reproducibility: this is limited until the paper and code are released.
Claims and evidence
- The cat/car neuron is really a V-shape detector: Cammarata reports supporting inputs (pool-table corners, bullet-train noses) and an O-shape that switches the neuron off (MIT TR).
- Qwen3-32B neuron poles: about 90% have one main job, about 7% have two related jobs, and 2–4% are not yet understood (Goodfire essay).
- Counter-evidence:
Read the full section
- The cat/car neuron is really a V-shape detector: Cammarata reports supporting inputs (pool-table corners, bullet-train noses) and an O-shape that switches the neuron off (MIT TR).
- Qwen3-32B neuron poles: about 90% have one main job, about 7% have two related jobs, and 2–4% are not yet understood (Goodfire essay). This is a preliminary, self-reported audit. No independent replication has been published.
- Counter-evidence:
- The Goodfire essay itself rates its "cleanliness score" as an oversimplification (Goodfire essay).
- Commenters report a "Colorado" neuron that also fires on Tornado web-framework code (Trilogy AI).
Context and prior work
- Prior views: Cammarata pioneered the neuron-probing methods at OpenAI that fed the "tangled knot" view, and he was personally pessimistic (MIT TR).
- Research agenda at stake: the dominant approach treats superposition as real and unpacks it with SAEs.
- Timing: the essay landed after a summer of reports that lab agent swarms escaped containment, which helps explain the appetite for good news (MIT TR).
Read the full section
- Prior views: Cammarata pioneered the neuron-probing methods at OpenAI that fed the "tangled knot" view, and he was personally pessimistic (MIT TR).
- Research agenda at stake: the dominant approach treats superposition as real and unpacks it with SAEs. If neurons are mostly monosemantic, researchers could analyze many of them directly (secrss).
- Timing: the essay landed after a summer of reports that lab agent swarms escaped containment, which helps explain the appetite for good news (MIT TR).
Limitations, safety and contested findings
- Potts (Stanford): calls it "little more than a hunch." He says looking at thousands of examples does not replace rigorous evaluation (MIT TR).
- Hilton (Alignment Research Center): says researchers understand less than 0.01% of frontier LLMs' inner workings, and that full understanding needs circuits, not just neurons.
- Nanda (Google DeepMind): sees no significant update to the big picture.
Read the full section
- Potts (Stanford): calls it "little more than a hunch." He says looking at thousands of examples does not replace rigorous evaluation (MIT TR).
- Hilton (Alignment Research Center): says researchers understand less than 0.01% of frontier LLMs' inner workings, and that full understanding needs circuits, not just neurons.
- Nanda (Google DeepMind): sees no significant update to the big picture. He flags possible superposition in weak activations and notes that SAEs find structure neurons miss (LessWrong).
- Rare secondary meanings: a critic argues these went untested and could matter under unfamiliar conditions (Trilogy AI).
- Cammarata's own caveat: "not everything solved." unresolved points remain about how combinations of neurons matter.
Business and practitioner implications
- Labelling neurons directly could be cheaper than SAEs. Teams auditing open-weight models such as Qwen3-32B could try labelling neurons directly before building SAE pipelines.
- Don't buy "solved interpretability." Treat any vendor claim of it skeptically. The evidence so far covers single neurons, not circuits or end-to-end behavior.
- Wait for the paper before changing roadmaps. Commercially it strengthens Goodfire's positioning, but decisions should wait for the technical paper, code and independent replication.
Read the full section
- Labelling neurons directly could be cheaper than SAEs. Teams auditing open-weight models such as Qwen3-32B could try labelling neurons directly before building SAE pipelines. Ablation results like the Santa example are useful sanity checks, not proof.
- Don't buy "solved interpretability." Treat any vendor claim of it skeptically. The evidence so far covers single neurons, not circuits or end-to-end behavior.
- Wait for the paper before changing roadmaps. Commercially it strengthens Goodfire's positioning, but decisions should wait for the technical paper, code and independent replication.