ModelsArchitectures & capability
Paper Instruments releases Ultramarine 1 Flash, a GLM-5.3 Flash model trained with rubric-graded reinforcement learning for office work
Paper Instruments has released Ultramarine 1 Flash, a GLM-5.3 Flash adapter trained with rubric-graded reinforcement learning to produce spreadsheets, slide decks and memos. Its main technique, weighted yes-or-no grading criteria that each test one condition, can be reused with other agents. The weights are still pending.
According to arXiv 2606.29920 (Jun 2026) [8]: Paper Instruments releases Ultramarine 1 Flash, a GLM-5.3 Flash model trained with rubric-graded reinforcement learning for office work.
SOURCES[8] Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic… arxiv.org
AI-assisted, not human-edited. Methodology Corrections Who owns this site
- Original source
- r/AISEOInsider (via last30days) article, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- Studies
- arXiv 2606.29920 (Jun 2026) [8] date from arXiv ID
- When
- October 17, 2026 article published
- Covered
- Oct 18 edition
- Evidence
- 9 sources across 9 sites
- 1How the Paper Instruments AI Method Fixes Any AI Agent)reddit.com
- 2Linked video: How the Paper Instruments AI Method Fixes…youtube.com
- 3Ultramarine 1 Flash: Scaling Non-Verifiable RL — Paper…paperinstruments.com
- 4AICoder: Paper Instruments releases Ultramarine 1 Flashaicoder.com
- 5Daanish Khazi LinkedIn postlinkedin.com
- 6Doomers.ai launch trackerdoomers.ai
QuietHow it’s measured
YouTube video · 451 views · 3 likes
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
Ultramarine 1 Flash is a rank-64 LoRA adapter on GLM-5.3 Flash. On Paper Instruments' own office-work suite it scored 59.3 at $0.13 per task. Claude Fable 5 scored 57.8 at $8.19. These are vendor-reported numbers from a private benchmark that no one has reproduced. [3] [4]
The grading method makes each rubric criterion test one condition and name the facts or thresholds needed to check it. Each criterion is graded yes or no and weighted by importance. This avoids criteria so strict they reject valid alternative wording, and criteria so vague they pass incomplete answers. Commentary argues teams can apply this to existing agents without changing models. [3] [2]
Training ran in 88 simulated office environments covering finance, legal, consulting and operations. Tasks were generated synthetically and screened for realism, leaked answers and reward hacking. The Feather harness rejects edits to files that changed after they were read, keeps a to-do list through context compaction, and trains with a system prompt of about 50 tokens. [3] [2]
There are trade-offs. Coding scores dipped slightly: Terminal-Bench 2.1 fell from 84.3 to 83.2, and DeepSWE 1.1 from 63.4 to 61.5. The model handles text only. The low per-task price means running a job several times and keeping the best rubric-graded output could still cost less than one premium-model run. [4] [3] [2]
Ultramarine scored 59.3 against Claude Fable 5's 57.8, at $0.13 versus $8.19 per task. Implication: if this holds on real workloads, cheap repeated runs graded by well-written rubrics could make document automation affordable.
Read the full assessment
Teams should still test on in-house documents first.
Executive brief
Paper Instruments says a five-person team spent about $300,000 post-training an open model, Ultramarine 1 Flash. On the company's own office-work benchmark, the model scored 59.3 at $0.13 per task. Claude Fable 5 scored 57.8 at $8.19 per task (Paper Instruments). That benchmark is not public, and no one has independently reproduced the result (AICoder). The Reddit post and its linked video are commentary that sells a paid community. Their useful point is that the paper's rubric-writing method works with any agent. Switching models is not required.
What changed and event timeline
Ultramarine 1 Flash launches
Paper Instruments released a post-trained version of GLM-5.3 Flash for spreadsheet, slide-deck and memo work, along with a technical write-up on reinforcement learning for tasks with no answer key.
CEO announcement spreads
CEO Daanish Khazi announced the release on X, stressing three numbers: a frontier-level comparison, a five-person team and a $300,000 budget. One tracker logged unusually high engagement for the post.
More detail
Trade coverage flags caveats
AICoder reported that the headline cost comparison depends on an internal benchmark nobody has reproduced. It also noted small drops in coding scores.
Commentary repackages the method
The Reddit post and its video call the rubric approach a "judgment machine" and recommend it for Claude Code and other agents. Both also promote a paid coaching community.
More detail
Capabilities and access
- Model: Ultramarine 1 Flash, a LoRA (rank 64) adapter on GLM-5.3 Flash with a 131K-token context window (Paper Instruments).
- Weights are promised in about two weeks (AICoder).
- Price: $0.15 per million input tokens and $0.50 per million output tokens (daily.dev).
Read the full section
- Model: Ultramarine 1 Flash, a LoRA (rank 64) adapter on GLM-5.3 Flash with a 131K-token context window (Paper Instruments). AICoder reports it can run up to 250 agent turns (AICoder).
- Access: an OpenAI-compatible API, live now. Weights are promised in about two weeks (AICoder).
- Price: $0.15 per million input tokens and $0.50 per million output tokens (daily.dev).
- Text only: the model has no vision, by design (Paper Instruments).
- Apps: the video says a Feather app for Apple Silicon Macs is opening from a waitlist (video).
Technical analysis for researchers and developers
The following details come from the company's write-up (Paper Instruments):
Read the full section
The following details come from the company's write-up (Paper Instruments):
- Rubrics: each grading criterion should test one condition and name the facts, thresholds and context needed to check it. Criteria are graded yes or no and weighted by importance. The goal is to avoid rejecting valid alternative answers while still catching plausible errors.
- Training environments: 88 simulated office environments across 19 practice areas: finance (34), legal (15), consulting (28) and operations (11). Each practice area allows roughly 10⁷–10¹⁰ task configurations. Tasks are generated synthetically and filtered for grounding, realism and reward hacking.
- RL objective: a modified CISPO objective with importance-sampling ratios clipped at 5, mean-centered advantages, asynchronous updates and a length penalty.
- Feather harness: file reads take a content fingerprint so stale edits are rejected, and a to-do list survives context compaction. The training system prompt is about 50 tokens, against 1,800–4,400 in the production harnesses studied.
- Paper Office: forks of the docx, pptx and xlsx Python libraries that validate edits so Office files don't break silently.
Claims and evidence
- Headline result: 59.3 at $0.13 per task versus Claude Fable 5 at 57.8/$8.19 and GPT-5.6 Sol at 55.1/$2.09 (Paper Instruments).
- Other benchmarks: an average gain of 4.1 points over the base model across six benchmarks, with 6.5% fewer output tokens.
- Cost and team: the $300,000 cost and five-person team are the CEO's own figures (AICoder).
Read the full section
- Headline result: 59.3 at $0.13 per task versus Claude Fable 5 at 57.8/$8.19 and GPT-5.6 Sol at 55.1/$2.09 (Paper Instruments). This is vendor-reported and measured on a private suite (AICoder). The video's "60 times cheaper" figure comes from these per-task costs (video).
- Other benchmarks: an average gain of 4.1 points over the base model across six benchmarks, with 6.5% fewer output tokens. Scores include JobBench 62.9 and FrontierFinance 53.3 (Paper Instruments).
- Cost and team: the $300,000 cost and five-person team are the CEO's own figures (AICoder).
- Independent corroboration: none was found in the reviewed sources.
Context and prior work
Reinforcement learning has mostly scaled on tasks a program can check automatically, such as code. The paper says coding takes the largest share of RL training across the industry (video). Research on rubric-based grading already shows that LLM judges and auto-generated rubrics are unreliable in agentic settings (arXiv 2606.29920; HackerNoon). Ultramarine's contribution is to apply rubric rewards at scale with an open base model.
Limitations, safety and contested findings
- Coding regressed slightly: Terminal-Bench 2.1 fell from 84.3 to 83.2, and DeepSWE 1.1 from 63.4 to 61.5 (AICoder).
- Chart presentation is contested: commenters called the charts cherry-picked, saying stronger competitors appear only in cost plots (daily.dev).
- Openness is unclear: sources disagree on what is released. One coverage summary says training code will be published (AICoder).
Read the full section
- Coding regressed slightly: Terminal-Bench 2.1 fell from 84.3 to 83.2, and DeepSWE 1.1 from 63.4 to 61.5 (AICoder).
- Chart presentation is contested: commenters called the charts cherry-picked, saying stronger competitors appear only in cost plots (daily.dev).
- Openness is unclear: sources disagree on what is released. One coverage summary says training code will be published (AICoder). A read of the company write-up found no release commitment in it.
- Reward hacking: the company says it kept adding new checks each time it caught the model gaming the score (video).
- Name errors in commentary: the CEO's name is garbled in the Reddit text and the auto-captions.
Business and practitioner implications
- Rubrics work with any agent: writing one criterion per idea, each tied to named facts, can be adopted without changing models.
- Cheap models allow best-of-N: at about $0.13 per task, running a job five times and keeping the best rubric-graded output costs less than one run on a premium model (video).
- Test on your own work first: before switching, compare it on in-house documents, and leave out work that depends on charts or images.
Read the full section
- Rubrics work with any agent: writing one criterion per idea, each tied to named facts, can be adopted without changing models. The video gives sales-call examples such as "names the lead's main worry" (video). This advice comes from commentary and has not been tested.
- Cheap models allow best-of-N: at about $0.13 per task, running a job five times and keeping the best rubric-graded output costs less than one run on a premium model (video).
- Test on your own work first: before switching, compare it on in-house documents, and leave out work that depends on charts or images.
The source trail.
Sources (9)
How the Paper Instruments AI Method Fixes Any AI Agent)
Feed excerpt only. A transcript of the video linked from this page is supplied (linked video auto captions; https://www.youtube.com/watch?v=k85bmBNAgN0); automatic text may contain errors.
reddit.comLinked video: How the Paper Instruments AI Method Fixes Any AI Agent)
linked video auto captions
www.youtube.com