ResearchPapers, evidence & method
Two Minute Papers Releases a Song Told From the Point of View of Gradient Descent
Two Minute Papers' latest episode is a song, not a paper explainer. In it, gradient descent describes stumbling downhill toward better answers. Its title echoes a documented research gap: deployed models mostly stay frozen and struggle to keep learning from new experience.
From the Two Minute Papers video of November 14, 2026 and cited sources: The episode announces no model, benchmark or result. [1]
SOURCES[1] AI Can Do Everything…Except This – Two Minute Papers… youtube.com
AI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- The episode announces no model, benchmark or result. [1]
- Original source
- Two Minute Papers video, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- November 14, 2026 video published
- Covered
- Nov 15 edition
- Evidence
- 9 sources across 9 sites; transcript sourced
- 1AI Can Do Everything…Except This – Two Minute Papers…youtube.com
- 2Loss of plasticity in deep continual learning – Amiiamii.ca
- 3What is Optimization? + Learning Gradient Descent – Rosettarosetta.to
- 4Surprise Video – What A Time To Be Alive! – Summifysummify.io
- 5Former OpenAI researcher says current AI models can't…the-decoder.com
- 6Measuring the Limits of Continual Learning for LLMs – ICML…icml.cc
RisingHow it’s measured
YouTube video · 38.8K views · 1.3K likes
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.

The episode announces no model, benchmark or result. Its lyrics have gradient descent narrate its own training: it falls, overshoots and undershoots, stalls on noisy gradients, and keeps improving a little at a time instead of reaching perfection. [1]
The 'except this' theme of learning steadily from one's own mistakes matches a known limitation. Peer-reviewed work shows that standard deep networks gradually lose plasticity in continual learning, until they learn no better than shallow networks. The authors propose continual backpropagation, which re-initialises rarely used units, as a fix. [2] [7] [9]
Newer evaluations suggest the gap remains open. A 2026 ICML workshop benchmark, ImprintBench, finds that both retrieval-based and training-based continual-learning systems still fail to absorb new experience reliably. A former OpenAI researcher has separately argued that models cannot update their beliefs after failing. [6] [5]
Peer-reviewed work shows standard deep networks gradually lose the ability to keep learning. A 2026 workshop benchmark finds that retrieval- and training-based fixes still fall short.
Read the full assessment
The implication: teams should not expect deployed models to improve from corrections without explicit retraining or memory layers.
Executive brief
Two Minute Papers' "AI Can Do Everything…Except This" (published 14 Nov 2026) comes with a transcript that contains no paper walkthrough at all. It is a song in which gradient descent tells its own story: it falls downhill, overshoots, stalls and keeps going, "not perfect, just better than yesterday" (0:55, 1:24). The video announces no model, benchmark or result. The "except this" theme, learning steadily from your own mistakes, matches a real, well-documented research gap. Deployed models mostly stay frozen after training, and peer-reviewed work shows that standard deep learning gradually loses the ability to keep learning (Amii).
What changed and event timeline
Channel explains gradient descent
Two Minute Papers #82 describes optimisation as nudging variables and keeping whichever changes improve the objective, using a cooking analogy ().
Nature paper names "loss of plasticity"
Dohare, Mahmood, Sutton and colleagues show that standard deep networks in continual learning slowly decline until they learn no better than shallow networks. They propose continual backpropagation as a fix ().
(ICML workshop): ImprintBench
Kalra, Goldblum and co-authors report that both retrieval-based and training-based continual-learning systems "still fall short" of learning reliably from new experience ().
An earlier musical episode
The channel released "Surprise Video – What A Time To Be Alive!", a song celebrating science, which set a precedent for non-paper episodes ().
Tworek flags learning from failure
Jerry Tworek, formerly of OpenAI and part of the o1/o3 reasoning work, said models have no good way to update their beliefs after failing, and called this a barrier to AGI ().
Capabilities and access
- No model, version, code or dataset is attached to the video. It is a free YouTube upload by Two Minute Papers.
- Related research artefacts that can actually be examined:
- continual backpropagation, described in the Nature paper (Amii);
Read the full section
- No model, version, code or dataset is attached to the video. It is a free YouTube upload by Two Minute Papers.
- Related research artefacts that can actually be examined:
- continual backpropagation, described in the Nature paper (Amii);
- the ImprintBench evaluation framework (ICML 2026).
Technical analysis for researchers and developers
- Read technically, the lines map onto known behaviour: noisy stochastic gradients (2:02), learning-rate overshoot and undershoot (2:21), and plateaus.
- Dohare et al. argue that gradient-based methods alone cannot sustain learning.
- ImprintBench tests whether new information is genuinely absorbed or only memorised. Scenarios include news events, API changes and changing user histories (ICML 2026).
Read the full section
- The lyrics as optimisation. Read technically, the lines map onto known behaviour: noisy stochastic gradients (2:02), learning-rate overshoot and undershoot (2:21), and plateaus. This is an interpretation; the video does not explain it.
- Plasticity. Dohare et al. argue that gradient-based methods alone cannot sustain learning. Their fix adds a random, non-gradient step: units that are barely used get re-initialised (Amii).
- Evaluation. ImprintBench tests whether new information is genuinely absorbed or only memorised. It covers six skills: acquiring information, temporal reasoning, resolving references, multi-hop reasoning, spotting implicit relevance, and recognising the limits of one's own knowledge. Scenarios include news events, API changes and changing user histories (ICML 2026).
Claims and evidence
- "Standard deep learning loses plasticity." This is a peer-reviewed result shown on ImageNet and on reinforcement-learning tasks (Nature via academic newsletter).
- "Current continual-learning systems fall short." The reviewed abstract gives no numbers (ICML 2026).
- "Models can't learn from failure." This is one practitioner's opinion, stated on a podcast (The Decoder).
Read the full section
- "Standard deep learning loses plasticity." This is a peer-reviewed result shown on ImageNet and on reinforcement-learning tasks (Nature via academic newsletter).
- "Current continual-learning systems fall short." This comes from a workshop poster, a lower bar than a full conference paper. The reviewed abstract gives no numbers (ICML 2026).
- "Models can't learn from failure." This is one practitioner's opinion, stated on a podcast (The Decoder).
- That the video is specifically about continual learning. This is inference from the title and lyrics. No coverage confirms it.
Context and prior work
- Two Minute Papers, run by Károly Zsolnai-Fehér, usually explains papers (Grokipedia). Musical episodes are rare but have happened before (Summify).
- Earlier press coverage presented the plasticity result as AI "losing the ability to learn": neurons go dead after repeated retraining, and random reactivation partly restores learning (CO/AI).
Limitations, safety and contested findings
- The video transcript is auto-generated captions of song lyrics. It contains no empirical claims, and some lines are garbled (for example around 3:44).
- Continuous belief updating is not the only proposed route. Persistent external memory is another, though ImprintBench finds that retrieval-based approaches also fall short (ICML 2026).
- Continual backpropagation still needs testing at larger scale before real-world use (CO/AI).
Read the full section
- The video transcript is auto-generated captions of song lyrics. It contains no empirical claims, and some lines are garbled (for example around 3:44).
- Continuous belief updating is not the only proposed route. Persistent external memory is another, though ImprintBench finds that retrieval-based approaches also fall short (ICML 2026).
- Continual backpropagation still needs testing at larger scale before real-world use (CO/AI).
- No other coverage of this specific video was found.
Business and practitioner implications
- Assume models are frozen. Deployed models will not learn from your corrections on their own.
- Agents tend to stall once their first approach fails (The Decoder). Design human escalation paths and retry strategies accordingly.
- Vendor claims about "learning". Ask for results on benchmarks that separate genuine absorption of new information from memorisation, as ImprintBench does.
Read the full section
- Assume models are frozen. Deployed models will not learn from your corrections on their own. Budget for retraining, retrieval or memory layers, and test that they actually work (ICML 2026).
- Agent reliability. Agents tend to stall once their first approach fails (The Decoder). Design human escalation paths and retry strategies accordingly.
- Vendor claims about "learning". Ask for results on benchmarks that separate genuine absorption of new information from memorisation, as ImprintBench does.