AgentsAutonomy & tool use
Microsoft open-sources ThinkingBox RL training pipeline; 27B Qwen model's tasks passed on all 20 attempts rise from 38 to 107
Microsoft released an MIT-licensed reinforcement learning pipeline that rewards agents for leaving databases in the correct final state. Fifty training steps made a 27B Qwen model far more consistent on the ThinkingBox benchmark, though slightly fewer tasks were solved at least once.
From the Hugging Face article of October 22, 2026 and cited sources: The reward reuses the benchmark's own grading. [1] [6] [7]
SOURCES[1] Turning Database Grading into RL Rewards (HF blog) huggingface.co[6] microsoft/thinkingbox-training (GitHub) github.com[7] DAPO: An Open-Source LLM RL System at Scale huggingface.co
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- Original source
- Hugging Face article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- Studies
- arXiv 2504.13837 (Apr 2025) [2] date from arXiv ID
- When
- October 22, 2026 article published
- Covered
- Oct 23 edition
- Evidence
- 8 sources across 5 sites
- 1Turning Database Grading into RL Rewards (HF blog)huggingface.co
- 2Does RL Really Incentivize Reasoning Capacity……arxiv.org
- 3One Success Isn't Reliability: Thinkingbox…arxiv.org
- 4The Agent Said It Was Done. The Database Disagreed.…huggingface.co
- 5Runtimewire: Microsoft releases an agent benchmark…runtimewire.com
- 6microsoft/thinkingbox-training (GitHub)github.com
Reach: not measured yet
The reward reuses the benchmark's own grading. A rollout scores 1 or 0 depending on checks of the final backend state, and a minority of narrow rubric checks use an LLM judge. Training tasks are kept only if the base model solves them between 8% and 90% of the time. This follows DAPO's practice of dropping groups that give GRPO no learning signal. [1] [6] [7]
On the held-out benchmark, Qwen3.8-27B at step 50 went from 38 to 107 of 507 tasks passed on all 20 attempts. Average success rose from 51.70% to 60.78%. Tasks solved at least once fell from 453 to 446. This fits earlier research finding that RL sharpens a model's consistency rather than widening what it can solve. [1] [2]
Average accuracy can hide differences in reliability. GPT-5.6 Sol beats the RL-trained Qwen on pass@1 but completes fewer tasks on all 20 attempts (82 vs. 107). The estimated serving cost per fully reliable task fell from $24.36 to $8.00, but that index leaves out training and task-probing costs. [1] [4]
repeated-run consistency improved sharply on unseen domains, while coverage narrowed slightly.
Read the full assessment
Implication: teams whose agents change production records could use executable state checks as rewards, but should track success across all attempts, not just average accuracy.
Executive brief
Microsoft says 50 steps of reinforcement learning nearly tripled the number of benchmark tasks a 27B open model completes correctly on every one of 20 attempts, from 38 to 107 of 507 (HF blog). The reward came from the same check the ThinkingBox benchmark uses to grade agents: whether the final database state is correct. Training used tasks kept separate from the benchmark. Average success rose from 51.70% to 60.78%, but the share of tasks solved at least once fell slightly. The model became more consistent, not more broadly capable. All results so far come from Microsoft and its co-authors. No independent replication of the training result has been published.
What changed and event timeline
Research on RL "sharpening" published
Yue et al. found that RL-trained models score better at small k but that base models score higher pass@k when k is large. The ThinkingBox team cites this to explain its own coverage loss.
ThinkingBox paper posted
The paper describes 507 stateful business workflows across five domains and evaluates 12 systems. The best scored 65.36% pass@1 but only 25.25% pass^20.
First blog post expands the leaderboard
Eighteen models were run 20 times on each task. Claude Opus 5.5 led pass@1 at 67.16% and completed 241 tasks on all 20 attempts.
Trade-press coverage
Coverage focused on the benchmark's idea of grading the records an agent leaves behind rather than its transcript. It did not evaluate the claims independently.
Training stack released
Microsoft published the RL pipeline under the MIT license, along with Qwen3.8-27B GRPO results on the held-out benchmark.
More detail
Capabilities and access
- Model and checkpoint: full-parameter RL on Qwen3.8-27B, evaluated at the step-50 checkpoint. The blog does not say whether the trained weights are released (HF blog).
- Code: microsoft/thinkingbox-training covers task selection, rollouts, rewards and a patched integration with the Verl 0.9.0 RL library.
- Data: the repository includes 39 public sample tasks.
Read the full section
- Model and checkpoint: full-parameter RL on Qwen3.8-27B, evaluated at the step-50 checkpoint. The blog does not say whether the trained weights are released (HF blog).
- Code: microsoft/thinkingbox-training covers task selection, rollouts, rewards and a patched integration with the Verl 0.9.0 RL library. It needs Linux, Python 3.12, and 3 nodes of 8 H100 GPUs for the reference run.
- Data: the repository includes 39 public sample tasks. The 187-task training pool is private, so users must bring their own authorized tasks that do not overlap with the benchmark.
Technical analysis for researchers and developers
- Reward: binary (1 or 0), based on checks of the final backend state.
- Task selection ("offline policy-relative difficulty"): each candidate task is run repeatedly against the untrained model.
- Training setup: token-mean GRPO with 18 groups of 8 rollouts per step, learning rate 1e-6.
Read the full section
- Reward: binary (1 or 0), based on checks of the final backend state. A minority of narrow rubric checks call an LLM judge (GPT-5 Chat). Each rollout gets a fresh backend, an isolated tool session, and an LLM-simulated user (HF blog).
- Task selection ("offline policy-relative difficulty"): each candidate task is run repeatedly against the untrained model. The success rate is estimated with a Beta(k+½, n−k+½) model (a Jeffreys prior). Tasks the model almost never solves (below 8%) or almost always solves (above 90%) are dropped, because GRPO learns nothing from groups where every rollout gets the same reward.
- Training setup: token-mean GRPO with 18 groups of 8 rollouts per step, learning rate 1e-6. Groups where all rollouts scored the same were replaced, following DAPO's dynamic sampling (DAPO).
- Implementation detail: training and evaluation share the code that parses the model's output, so the reward measures the same agent that is later scored.
Claims and evidence
| Claim | Base → RL-trained | Source |
| Tasks passed on all 20 attempts | 38 → 107 | HF blog |
| pass@1 (average success) | 51.70% → 60.78% | same |
| pass@20 (solved at least once) | 453 → 446 tasks | same |
| Estimated cost per task passed 20/20 | $24.36 → $8.00 | same; this is a relative serving index, not actual bills |
- All figures are from the authors. No third-party rerun has been published, and the training pool is private. The trade-press reviews cover only the benchmark (hyper.ai).
Context and prior work
- The benchmark paper evaluated 12 older models. The blogs report 18, including newer ones (paper; first post).
- The pattern of better consistency with slightly narrower coverage matches the "sharpening" account in Yue et al. (arXiv).
- Filtering out tasks that give no learning signal builds on DAPO's dynamic sampling, which is also built on Verl (DAPO).
Limitations, safety and contested findings
- Limited transfer claim: the benchmark uses domains and databases not seen in training, but it shares the framework, the user-simulator machinery, some tools and the task style.
- Uneven gains: pass@20 fell in retail, travel and auto insurance. Auto insurance lost the most, at −7.0 points.
- Simulated user varies by source: the paper uses GPT-5.4-mini as the simulated user and notes possible bias toward that model family.
Read the full section
- Limited transfer claim: the benchmark uses domains and databases not seen in training, but it shares the framework, the user-simulator machinery, some tools and the task style. The authors themselves say this limits the transfer claim (HF blog).
- Uneven gains: pass@20 fell in retail, travel and auto insurance. Auto insurance lost the most, at −7.0 points.
- Simulated user varies by source: the paper uses GPT-5.4-mini as the simulated user and notes possible bias toward that model family. Training rollouts used GPT-5 Chat (paper).
- Synthetic tasks: the paper says its tasks are synthetic reconstructions and does not claim they represent real enterprise workloads.
- Inconsistent failure statistic: the share of failures attributed to tool handling is 77.5% in the paper and 79.85% in the first blog, which used a different set of models.
Business and practitioner implications
- Look past pass@1: for agents that change production records, check repeated-run consistency.
- Smaller models can catch up on reliability: a 27B model trained with checks you can execute moved close to frontier models on estimated cost per reliably completed task ($8.00 vs. GPT-5.4's $6.80).
- The bottleneck is task data: reproducing this requires your own tasks that are separate from the benchmark and that produce mixed results for your model.
Read the full section
- Look past pass@1: for agents that change production records, check repeated-run consistency. In the blog's table, GPT-5.6 Sol beats the RL-trained Qwen on pass@1 but completes fewer tasks on all 20 attempts (82 vs. 107) (HF blog).
- Smaller models can catch up on reliability: a 27B model trained with checks you can execute moved close to frontier models on estimated cost per reliably completed task ($8.00 vs. GPT-5.4's $6.80). That index excludes training and task-probing costs.
- The bottleneck is task data: reproducing this requires your own tasks that are separate from the benchmark and that produce mixed results for your model.