AgentsAutonomy & tool use
Asana cuts browser-agent costs 76x in tests, mostly by fixing prompt caching before switching to GPT-6.1 Sol
Asana's StackAI division cut a browser agent's per-run cost from at least $36.21 to about $0.47 in a controlled study. Fixing how the agent cached page history delivered 29x; moving to GPT-6.1 Sol added roughly 2.6x.
Reach: not measured yet
AT Pulse research deskPublished · 5 min read · AI-assisted analysisMost of the savings came from a caching bug, not the model switch. The agent cached only its fixed system prompt and tool definitions, so it resent its growing history of page text and screenshots at full price on every call. The team added a cache marker on the latest tool result, kept the history append-only and pruned screenshots in batches. Those changes cut costs about 29x on the existing production model. [4] [2] [1]
GPT-6.1 Sol's extra 2.6x is largely about cached-input pricing. Sol charges $0.10 per million cached input tokens, compared with $1 for GPT-6 Astra. After the fix, cache reads made up 89% of Sol's input. For long-context agents, the cached-input price can matter more than the headline per-token rate. [3] [4] [2]
GPT-6 Astra running in Codex did the optimization work. It audited the codebase, added logging to requests, refactored the workflows, ran the experiments and analyzed the traces, while humans set the goals and reviewed the conclusions. That explains why OpenAI's materials credit both Astra and Sol: Astra did the engineering, and Sol was the model the agent ran on. [4] [1] [5]
in Asana's own unaudited study, cache design drove most savings. Implication: teams running long-context agents should audit caching and compare cached-input prices before switching models, and measure cost per completed task rather than per token.
Executive brief
Most of Asana's headline "76x cheaper" result came from fixing how its browser agent cached data, not from switching models. Fixing the caching cut costs 29x on the existing production model. Moving to GPT‑6.1 Sol added only another 2.6x (OrcaRouter). OpenAI's feed excerpt gives credit to GPT‑6 Astra in Codex. The headline names GPT‑6.1 Sol (OpenAI). Both are right: Astra did the engineering work, and Sol was the model the agent ran on. The test was a controlled study of 144 runs on a demo book catalog, not a production benchmark.
What changed and event timeline
OpenAI ships GPT‑6.1 Sol
Released at one-fifth of GPT‑6 Astra's input and output prices, with cached input at $0.10 per million tokens. The benchmarks were run by OpenAI ().
StackAI publishes its browser-agent study
StackAI is a division of Asana. Its write-up describes 144 runs plus a 12-run follow-up across four models (; date per).
OpenAI publishes the customer story
The excerpt says GPT‑6 Astra in Codex made the agent 76x cheaper and 5x faster "in tests" ().
The first commentary flags gaps
Ken Ashe points out the mismatch between Astra and Sol in OpenAI's wording, and says the excerpt omits the baseline, the task set and success rates ().
The cost breakdown appears
OrcaRouter splits the 76x into 29x from the workflow fix and 2.6x from the model, and notes that the figures have not been independently audited ().
Capabilities and access
- GPT‑6.1 Sol costs $2 per million input tokens, $0.10 per million cached input tokens and $10 per million output tokens.
- GPT‑6 Astra costs $10 per million input tokens, $1 per million cached input tokens and $50 per million output tokens (same source).
- Roles in the study: Astra, running in Codex, audited the codebase, added logging to requests, refactored the workflows, ran the experiments and analyzed the traces.
Read the full section
- GPT‑6.1 Sol costs $2 per million input tokens, $0.10 per million cached input tokens and $10 per million output tokens. It is available in the API and in ChatGPT Work and Codex. An Ultrafast tier costs 6x the standard price (VentureBeat).
- GPT‑6 Astra costs $10 per million input tokens, $1 per million cached input tokens and $50 per million output tokens (same source).
- Roles in the study: Astra, running in Codex, audited the codebase, added logging to requests, refactored the workflows, ran the experiments and analyzed the traces. Humans set the goals and reviewed the conclusions (StackAI).
Technical analysis for researchers and developers
- The bug: The agent cached its fixed system prompt and tool definitions.
- The fixes:
- Put a cache marker on the latest tool result.
Read the full section
- The bug: The agent cached its fixed system prompt and tool definitions. It did not cache the growing history of page text and screenshots, so every call resent that history at full price. On the models tested, cached reads cost 0.05x to 0.1x the normal input price (OpenAI via search excerpt).
- The fixes:
- Put a cache marker on the latest tool result.
- Keep the history append-only.
- Prune screenshots in batches (keep up to 20, then cut to 1), so about 19 calls in a row can reuse the cache.
- Raise the history budget from 120,000 to 480,000 characters.
(StackAI)
- Evaluation: Six caching and history policies were tested on four models, three runs per condition, at two budget levels. Answers were scored against references prepared separately (same source).
- Reproducibility: Three of the four models are anonymized as Models A, B and C, so outside readers cannot replicate the comparison (OrcaRouter).
Claims and evidence
- 76x cheaper and 5x faster: About $0.47 and roughly 4 minutes per run.
- Where the savings came from: 29x from the workflow fix on Model B, 2.6x from switching to GPT‑6.1 Sol (same source).
- Cache reads: 89% of input on GPT‑6.1 Sol (StackAI).
Read the full section
- 76x cheaper and 5x faster: About $0.47 and roughly 4 minutes per run. The original production setup cost at least $36.21 and took at least 22.5 minutes (OrcaRouter).
- Where the savings came from: 29x from the workflow fix on Model B, 2.6x from switching to GPT‑6.1 Sol (same source).
- Cache reads: 89% of input on GPT‑6.1 Sol (StackAI).
- Who reported these numbers: All figures come from Asana and OpenAI. No independent replication or audit was found.
- Speed of the study: StackAI's CTO, Frank Hidalgo, says the week-long study would have taken months by hand. This is a self-assessment (same source).
Context and prior work
- GPT‑6.1 Sol launched about a week after GPT‑6 Sol and is positioned as close to Astra at lower cost (The New Stack).
- OpenAI's own evaluations put it within 2.1 points of Astra on OSWorld 2.0 and matching Astra on DeepSWE v1.1 (VentureBeat).
- The Asana study shows how much the cached-input price matters. Sol's cached price is one-tenth of Astra's, which helps explain its additional 2.6x.
Read the full section
- GPT‑6.1 Sol launched about a week after GPT‑6 Sol and is positioned as close to Astra at lower cost (The New Stack).
- OpenAI's own evaluations put it within 2.1 points of Astra on OSWorld 2.0 and matching Astra on DeepSWE v1.1 (VentureBeat).
- The Asana study shows how much the cached-input price matters. Sol's cached price is one-tenth of Astra's, which helps explain its additional 2.6x.
Limitations, safety and contested findings
- Reliability depends on the history budget: At the 120,000-character budget, only 3 of 18 GPT‑6.1 Sol runs produced answers.
- Small samples: Three or four runs per condition can show broad patterns but cannot separate conditions that differ by a few percent (StackAI).
- Narrow task: One demo task (six fields from 32 books). Pruning made little difference on shorter tasks, about 1.2x (same source).
Read the full section
- Reliability depends on the history budget: At the 120,000-character budget, only 3 of 18 GPT‑6.1 Sol runs produced answers. At 480,000 characters, all runs succeeded (OrcaRouter).
- Small samples: Three or four runs per condition can show broad patterns but cannot separate conditions that differ by a few percent (StackAI).
- Narrow task: One demo task (six fields from 32 books). Pruning made little difference on shorter tasks, about 1.2x (same source).
- Model naming: OpenAI's excerpt credits Astra while its headline credits Sol (Ken Ashe).
Business and practitioner implications
- Check caching before changing models: Most of the savings came from cache design, which any team can check on its current provider.
- Measure cost per completed task, not per token. Also track retries and failure types (Ken Ashe).
- Compare cached-input prices when routing long-context agents.
Read the full section
- Check caching before changing models: Most of the savings came from cache design, which any team can check on its current provider.
- Measure cost per completed task, not per token. Also track retries and failure types (Ken Ashe).
- Compare cached-input prices when routing long-context agents.
- Coding agents can run optimization studies, but humans should set the goals and review the conclusions, as in this case.