ResearchPapers, evidence & method
Jack Clark moves his recursive self-improvement forecast to 2027, citing the new TasteVal research-taste benchmark
In Import AI 476, Anthropic co-founder Jack Clark says full recursive self-improvement now looks more likely in 2027 than 2028. He points to TasteVal, which finds that frontier models' experimental research taste doubles about every three months. The issue also covers lagging Western open-weight models.
From the Import AI article of October 12, 2026 and cited sources: Clark now expects full recursive self-improvement to be more likely in 2027 than 2028. [1] [2]
SOURCES[1] Import AI 476: Creativity and RSI; worse-than-China open… jack-clark.net[2] TasteVal: Measuring the Experimental Research Taste of AI… lesswrong.com
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Clark now expects full recursive self-improvement to be more likely in 2027 than 2028. [1] [2]
- Original source
- Import AI article, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 12, 2026 article published
- Covered
- Oct 13 edition
- Evidence
- 6 sources across 6 sites
- 1Import AI 476: Creativity and RSI; worse-than-China open…jack-clark.net
- 2TasteVal: Measuring the Experimental Research Taste of AI…lesswrong.com
- 3Reflection AI Beam (eesel)eesel.ai
- 4Mistral launches Large 4 (The Next Web)thenextweb.com
- 5Scale AI releases visual-reasoning benchmark… (RuntimeWire)runtimewire.com
- 6Mistral Large 4 launches, open weights late October…pasqualepillitteri.it
Reach: not measured yet
Clark now expects full recursive self-improvement to be more likely in 2027 than 2028. He had earlier named creativity as the main bottleneck, and he now points to progress in research taste. In TasteVal, a fixed coding agent runs the experiments, so the model being tested only designs them and interprets the results. On this setup, frontier models' taste doubled every 3.0 months (95% CI 1.7–5.0) since December 2025. Opus 5.5 reached a 2.30x compute multiplier (CI 1.15–4.37x) over the best human-expert attempt on each of 8 AI R&D tasks, at about one-thirtieth of the experts' cost per attempt. [1] [2]
Clark argues the US has largely left the open-weight frontier. Two new Western releases still trail Chinese models on several agentic benchmarks. Reflection's Beam (501B total, 23B active parameters) had no released weights as of October 8, and its waitlisted API offers 256K tokens of context instead of the advertised 1M. Clark cites Beam at 44.4 on DeepSWE 1.1, against 61 for GLM-5.3. Mistral Large 4 (1T total, 49B active) is in API preview, with weights due October 27. Mistral reports 62% on DeepSWE against GLM-5.3's 61%, a vendor figure that has not been independently tested. [1] [3] [4] [6]
Scale AI and Elorian's Humanity's Sixth Sense benchmark tests what images and video imply but don't show. The leading model, GPT-6-astra, scores 53.6%, against 93.1% for humans, and the median across 25 models is 30.9%. More reasoning effort lowered GPT-6-astra's score in some subdomains. Clark reads the results as evidence against the view that visual reasoning needs explicit world models. [1] [5]
TasteVal's results rest on eight tasks and have wide confidence intervals. If the trend holds, AI-designed experiments could cut research costs and support Clark's earlier RSI timeline.
Read the full assessment
Meanwhile, buyers who need open weights face a capability gap against Chinese models.
Executive brief
The most consequential claim in Import AI 476 is that Jack Clark, an Anthropic co-founder, now expects full recursive self-improvement (RSI) to be more likely in 2027 than 2028. He bases the change on TasteVal, a new benchmark that measures AI "research taste." It reports that frontier models' experimental taste has doubled every 3.0 months since December 2025, and that Opus 5.5 beats a baseline of 24 human experts (Import AI 476; LessWrong). The issue also covers two Western open-weight models, Reflection's Beam and Mistral Large 4, that trail Chinese models on several benchmarks. It also covers a visual-intuition benchmark where the best model scores 53.6% and humans score 93.1%.
What changed and event timeline
Reflection unveils Beam
Beam is a 501B mixture-of-experts model with 23B parameters active per token. Reflection promises Apache 2.0 weights later in October. As of October 8, the weights were not out and API access required a waitlist ().
Mistral announces Large 4 ("Le Chonk")
The model has 1T parameters, 49B of them active per token. An API preview is live and weights are due October 27 ().
TasteVal published
The benchmark measures how much experimental compute a model needs to reach a given score, compared with human experts. Opus 5.5 scores a 2.30x compute multiplier against the human baseline ().
Scale AI and Elorian release Humanity's Sixth Sense (HSS)
HSS tests what a visual scene implies but doesn't show. GPT-6-astra leads at 53.6%, against 93.1% for humans ().
Import AI 476 published
Clark pulls these items together, moves his RSI estimate to 2027, and adds pieces on AI and space and on federated training across Department of Energy (DOE) labs ().
Capabilities and access
- Beam (Reflection): 501B total / 23B active parameters, text-only, knowledge cutoff June 30, 2026.
- Mistral Large 4: 1T total / 49B active parameters, takes text and images in and returns text. API preview now, weights due October 27.
- Opus 5.5 and GPT-6-astra are the leading models on TasteVal and HSS respectively.
Read the full section
- Beam (Reflection): 501B total / 23B active parameters, text-only, knowledge cutoff June 30, 2026. A 1M-token context is advertised, but the waitlisted API offers 256K. The weights had not been released as of October 8 (eesel).
- Mistral Large 4: 1T total / 49B active parameters, takes text and images in and returns text. API preview now, weights due October 27. Developers, security firms and government agencies get a version with fewer restrictions (TNW; Pasquale Pillitteri).
- Opus 5.5 and GPT-6-astra are the leading models on TasteVal and HSS respectively.
Technical analysis for researchers and developers
- TasteVal method: A fixed coding agent carries out the experiments, so the model under test only designs them and interprets the results.
- HSS: 522 open-ended tasks over 288 images and 234 video clips. Across 25 models the median score is 30.9%.
- Beam: 52 layers that alternate local and global attention.
Read the full section
- TasteVal method: A fixed coding agent carries out the experiments, so the model under test only designs them and interprets the results. Each run is capped at 40 GPU-hours or 120 wall-clock hours. The human baseline is the best expert attempt on each of 8 AI R&D tasks (LessWrong).
- HSS: 522 open-ended tasks over 288 images and 234 video clips. Across 25 models the median score is 30.9%. More reasoning effort lowered GPT-6-astra's score in some subdomains (RuntimeWire).
- Beam: 52 layers that alternate local and global attention. Reinforcement-learning training ran four weeks on about 10.5K GB300 GPUs and produced more than 100M rollouts (eesel).
- DOE federated training: Federated pretraining ran across El Capitan (AMD MI300A) and H100 clusters while restricted data stayed at its home site. A separate fine-tuning run used about 1,700 GPUs across four supercomputers (Import AI 476).
Claims and evidence
- TasteVal results: taste doubles every 3.0 months (95% CI 1.7–5.0), Opus 5.5 reaches 2.30x (CI 1.15–4.37x), at about 1/30 of the experts' cost per attempt (LessWrong).
- Beam efficiency: Reflection says Beam matches GLM-5.2 on reasoning with 3–4× less inference compute.
- Clark cites DeepSWE 1.1 at 44.4 versus 61 for GLM-5.3 (Import AI 476). eesel reports SWE-bench Pro at 65.5 versus 62.1 for GLM-5.2 (eesel).
Read the full section
- TasteVal results: taste doubles every 3.0 months (95% CI 1.7–5.0), Opus 5.5 reaches 2.30x (CI 1.15–4.37x), at about 1/30 of the experts' cost per attempt (LessWrong). These come from the benchmark's own authors; no independent replication has been found.
- Beam efficiency: Reflection says Beam matches GLM-5.2 on reasoning with 3–4× less inference compute. This is a vendor claim, and the estimate excludes prompt prefill and attention overhead (eesel).
- Beam benchmark figures differ by source. Clark cites DeepSWE 1.1 at 44.4 versus 61 for GLM-5.3 (Import AI 476). eesel reports SWE-bench Pro at 65.5 versus 62.1 for GLM-5.2 (eesel).
- Mistral claims: DeepSWE 62% versus 61% for GLM-5.3 (TNW). Mistral says it scores 82% on a reproduce-and-patch vulnerability test, the highest of any model (Pillitteri). TNW notes Mistral gave no comparative figures for its cyber claims.
Context and prior work
- Clark had earlier named creativity as the key bottleneck for RSI, in Import AI 455.
- Clark argues the US has largely left the open-weight frontier, apart from Google's small Gemma models, because Meta stopped releasing strong Llama models.
- Reflection explicitly positions Beam as advancing a "Western open-weight frontier" (eesel).
Read the full section
- Clark had earlier named creativity as the key bottleneck for RSI, in Import AI 455. He now cites progress on math breakthroughs and research taste (Import AI 476).
- Clark argues the US has largely left the open-weight frontier, apart from Google's small Gemma models, because Meta stopped releasing strong Llama models.
- Reflection explicitly positions Beam as advancing a "Western open-weight frontier" (eesel).
Limitations, safety and contested findings
- TasteVal's authors list their own caveats: the tasks favor problems that are easy to verify and "hill-climb". The experts may underrepresent top researchers.
- HSS has two readings. Clark takes the scores as evidence against the view that visual reasoning needs explicit world models.
- Mistral's training compute is reported inconsistently: 3,800 GPUs in Import AI versus about 4,000 over two months in TNW.
Read the full section
- TasteVal's authors list their own caveats: the tasks favor problems that are easy to verify and "hill-climb"; the experts may underrepresent top researchers; choosing which problems to work on is not measured; and models got limited elicitation (LessWrong).
- HSS has two readings. Clark takes the scores as evidence against the view that visual reasoning needs explicit world models. Read the other way, the 40-point gap to humans points to weaknesses that more reasoning effort does not fix (RuntimeWire).
- Mistral's training compute is reported inconsistently: 3,800 GPUs in Import AI versus about 4,000 over two months in TNW.
- Mistral's less-restricted cyber variant raises dual-use questions.
Business and practitioner implications
- Open weights: Western open-weight options are improving but still trail Kimi K3, GLM-5.3 and DeepSeek on several agentic benchmarks.
- Beam: don't plan around it until the weights and technical report actually ship. Its API context limit (256K) is also below the advertised 1M.
- Security teams: Mistral is pitching self-hosted cyber work with fewer refusals. Its claims need independent evaluation first.
Read the full section
- Open weights: Western open-weight options are improving but still trail Kimi K3, GLM-5.3 and DeepSeek on several agentic benchmarks. Teams with sovereignty or compliance needs face a capability tradeoff.
- Beam: don't plan around it until the weights and technical report actually ship. Its API context limit (256K) is also below the advertised 1M.
- Security teams: Mistral is pitching self-hosted cyber work with fewer refusals. Its claims need independent evaluation first.
- R&D leaders: if TasteVal's trend holds, AI-run experiment design could cut research costs within months.
- Government: DOE's federated runs are about 100× below the frontier in scale, according to Clark.
The source trail.
Sources (6)
Import AI 476: Creativity and RSI; worse-than-China open weight models; AI goes to space
Article text retrieved; extracted text may omit tables or interactive elements.
jack-clark.net