While it will AI will greatly benefit society, it will also displace hundreds of millions of jobs. We need better benchmarks to help politicians and enterprises measure model performance and prepare for the displacement.

APEX turns 1 today. Last year, we launched the AI Productivity Index (APEX) to answer whether frontier AI models can do professional work. Since then, we’ve created new benchmarks while model capabilities progress more rapidly than even the boldest predictions. APEX remains the industry standard for evaluating frontier AI on economically valuable work. Here’s how APEX has evolved over the past year. 𝗢𝗰𝘁 𝟮𝟬𝟮𝟱: 𝗔𝗣𝗘𝗫 Mercor introduces APEX, our first AI benchmark for testing investment banking, law, consulting, and medicine. 𝗝𝗮𝗻 𝟮𝟬𝟮𝟲: 𝗔𝗣𝗘𝗫-𝗔𝗴𝗲𝗻𝘁𝘀 Built with partners @box and @harvey, APEX-Agents evaluates AI agents on long-horizon tasks in investment banking, consulting, and corporate law. 𝗠𝗮𝗿 𝟮𝟬𝟮𝟲: 𝗔𝗣𝗘𝗫-𝗦𝗪𝗘 Co-developed with @cognition, APEX-SWE assesses real production engineering across integration and observability. 𝗝𝘂𝗹 𝟮𝟬𝟮𝟲: 𝗔𝗣𝗘𝗫-𝗔𝗰𝗰𝗼𝘂𝗻𝘁𝗶𝗻𝗴 Built with @tryramp and @RampLabs, APEX-Accounting measures whether models can close the books. 𝗦𝗲𝗽 𝟮𝟬𝟮𝟲: 𝗔𝗣𝗘𝗫-𝗔𝗴𝗲𝗻𝘁𝘀 𝟭.𝟭 Our first major update to APEX-Agents includes newly audited tasks, an improved judge, and enhancements to ensure leaderboard accuracy. Thank you to all our partners who built these benchmarks with us. And, to the Mercor experts, professional bankers, lawyers, consultants, doctors, engineers, and accountants, whose judgement and expertise ensure these benchmarks are realistic. The rate of AI progress is only increasing. Mercor is committed to maintaining and extending our APEX family of benchmarks as the industry standard for informing decisions about the AI frontier and its ability to do professional work. See full leaderboards: mercor.com/apex/

New research: we hired 12 CPAs to complete four realistic accounting tasks, then had AI models from the past few years complete the same work. As of May 2025, accountants outperformed AI. Now they lose out to today's frontier models, which are near perfect on these tasks. 🧵

We compared 12 licensed accountants against frontier AI models on realistic accounting tasks. The models were faster and more accurate than every accountant in our study. 18 months ago, the best models scored below the average accountant's ~37%. Today, models ace the same tasks. This doesn’t mean accountants are replaceable. But it does suggest the job will change significantly, even if model progress stalled today. Read our full study: mercor.com/blog/human-baseli…

Gemini 4 Argon is the new #1 on APEX-Agents. 82.2% Pass@1 (#1) 87.9% Mean score (#1) It is the first model to exceed 80% Pass@1. It is +6.7 pts over the prior #1, Sonnet 5.5, and +14.4 pts over the best prior model from DeepMind, Gemini 3.7 Flash (67.8%). #𝟭 𝗶𝗻 𝗮𝗹𝗹 𝗔𝗣𝗘𝗫-𝗔𝗴𝗲𝗻𝘁𝘀 𝗱𝗼𝗺𝗮𝗶𝗻𝘀 Google says Argon delivers frontier performance in "enterprise knowledge work like legal and finance." On APEX-Agents, Gemini 4 Argon ranks first in every professional domain. Management consulting: 90.3% (#1) Investment banking: 80.9% (#1) Corporate law: 75.3% (#1) Consulting appears to be Argon's strongest domain, scoring 10.3 pts ahead of Opus 5.5. It is also the first model to score above 90% on any APEX-Agents leaderboard. 𝗧𝗼𝗸𝗲𝗻 𝘂𝘀𝗮𝗴𝗲 𝗯𝘆 𝗔𝗣𝗘𝗫-𝗔𝗴𝗲𝗻𝘁𝘀 𝗱𝗼𝗺𝗮𝗶𝗻 Argon used 2.6M tokens per attempt on average. The median attempt used 1.75M. Investment banking: 3.5M per attempt Corporate law: 2.7M Management consulting: 1.7M Almost all Gemini 4 Argon’s token usage comes from input. It used 2.61M input tokens per attempt compared with only 28.5K output tokens per attempt. That is about 90 input tokens for every output token. The agent reads a lot of files before composing a short answer. Congrats to @Google and @GoogleDeepMind. See full leaderboard: mercor.com/apex/apex-agents-…

Voice agents need to work for everyone, whatever language we speak, however our names are spelled, wherever we live, and whatever specialized words we use. Two new papers from @SierraPlatform's τ-voice team help put a number on that: τ-Elicitation: Benchmarking for multi-turn entity extraction in voice agents τ-Multilingual: Extends τ-voice to a multilingual settings Proud that Mercor's Victor Barres was a key contributor. Congrats to the whole team! Read the full papers. arxiv.org/abs/2609.13602v1 arxiv.org/abs/2609.35820v1

GPT-6.1 Sol scores 60.0% Pass@1 and 73.2% mean on APEX-Agents (#11), and 46.9% on APEX-SWE (#18). @OpenAI says it nearly matches GPT-6 Astra on professional work "at one-fifth of Astra's standard input and output token prices." That is consistent with what we’re seeing on APEX, within 4.7 points of Astra on Agents and 3.1 points on SWE. 𝗔𝗣𝗘𝗫-𝗔𝗴𝗲𝗻𝘁𝘀 GPT-6.1 Sol improves Pass@1 across all APEX-Agents domains compared with GPT-6 Sol. Corporate law: 63.4% (#11), up 7.2 Management consulting: 58.4% (#12), up 6.8 Investment banking: 58.1% (#10), up 2.9 That makes GPT-6.1 Sol OpenAI's #2 model on APEX-Agents, behind Astra and ahead of GPT-5.6 Terra (58.2%). It improves 5.7 points over GPT-6 Sol overall. On investment banking, GPT-6.1 Sol beats Astra (55.9%). It is now OpenAI's top model there. For corporate law, GPT-6.1 Sol trails Astra by 10 points on Pass@1, but its mean score is 85.3%. Only 4% of law runs score zero. It does most of the work and misses a rubric item or two. 𝗧𝗼𝗸𝗲𝗻𝘀 𝗮𝗻𝗱 𝗰𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝗰𝘆 0.50M tokens per attempt on APEX-Agents. 1.29M per attempt on APEX-SWE. On law, failing runs use 2.4x the tokens of passing runs. GPT-6.1 Sol passed 49% of APEX-Agents tasks on all 4 runs. On APEX-SWE, 39%. 70 Agents tasks were never solved in any run. 𝗔𝗣𝗘𝗫-𝗦𝗪𝗘 GPT-6.1 Sol improves slightly over the prior generation with 46.9% (#18), up 1.9 from GPT-6 Sol. The gain is all in Observability where it scored 32.3% (#19), increasing by 4.5. Integration was mostly flat at 61.5%, scoring slightly below GPT-6 Sol (62.3%) and GPT-6 Astra (62.0%). Congrats @OpenAI on the launch. See full leaderboards: mercor.com/apex/

Claude Sonnet 5.5 debuts at #1 on APEX-Agents and #2 on APEX-SWE. APEX-Agents: 75.5% Pass@1 (#1) 21 pt gain from from 54.5% for Sonnet 5 APEX-SWE: 66.4% Pass@1 (#2) 20 pt gain from 46.4% for Sonnet 5 Anthropic states Sonnet 5.5 at Max effort performs comparably to Opus 5.5. On APEX-Agents, it scores 2.0 pts higher (75.5% vs. 73.5%). APEX-Agents shows big improvement over Sonnet 5, where two domains gain more than 20 points. Investment banking: 77.2% Pass@1 (#1), up from 54.1% (+23.1 pts) Management consulting: 79.7% (#2), up from 47.8% (+31.9 pts) Corporate law: 69.6% (#5), up from 61.6% (+8.0 pts) In investment banking, Sonnet 5.5 takes over #1 from Gemini 3.7 Flash (71.3%). Consulting is the biggest jump over Sonnet 5, gaining +31.9 pts over the prior generation. On APEX-SWE, Sonnet 5.5 is 1.2 pts behind Opus 5.5 (67.6%). Integration: 69.3% (#1), up from 60.3% (+9.0 pts) Observability: 63.5% (tied #2), up from 32.5% (+31.0 pts) Sonnet 5 solved about 1 in 3 production debugging tasks, and now Sonnet 5.5 solves nearly 2 in 3. Congrats to @AnthropicAI on the launch. See full leaderboard: mercor.com/apex/

Claude Opus 5.5 is the new #1 on APEX-SWE. Overall, Opus 5.5 scores 67.6% Pass@1, a +3.9 point gain over the previous leader, Opus 5, at 63.7%. APEX-SWE measures observability, debugging from production telemetry, and integration, if a model can build an end-to-end system. Compared with Opus 5: Observability Opus 5.5: 69.8% (#1) Opus 5: 63.5% (#2) It gains +6.3 points. Opus 5.5 leads the board by 6.3 over Opus 5 and 10.8 over Fable 5.1. Integration Opus 5.5: 65.5% (#4) Opus 5: 64.0% (#9) Opus 5.5 improved at reading telemetry and finding faults, but building systems from scratch moved less. Congrats to @AnthropicAI. APEX-SWE leaderboard: mercor.com/apex/apex-swe-lea…

Mercor is committing $5M to a new AI Capabilities Fund. In conversations with researchers and enterprises, one challenge that comes up repeatedly is that many AI evals do not reflect how models are used. But to provide useful signal for eval and training, benchmarks need to capture actual workflows in representative environments. This will only become more difficult as models are used for more high-value, complex tasks. The Capabilities Fund supports researchers working on frontier eval challenges across a range of domains and data shapes, including: - Reward calibration and aligning grades with expert preferences - Creating high-quality realistic environments - Long-horizon workflows - Ambiguous requests and under-specified outcomes - Navigating social, temporal, and business context We will fund researcher time, API credits, and travel. We will also cover the cost to work with our network of 5M+ experts, as well as free use of our evals and analysis platform. Grants are available to independent researchers, non-profits, and academics. To submit an Expression of Interest, go to: mercor.com/careers/?ashby_ji…

OpenAI GPT‑6 Sol and Luna are on the APEX leaderboards. GPT-6 Sol scores 54.3% Pass@1 (#18) on APEX-Agents. That is +2.9 pts over GPT-5.6 Sol. GPT-6 Luna scores 44.3% Pass@1 (#29), +1.3 pts over GPT-5.6 Luna. On APEX-SWE, Luna scores 38.8% Pass@1, +6.7 pts. OpenAI reduced API prices for Sol and Luna by 50% compared with their GPT‑5.6 promotional pricing. GPT-6 Sol improves in finance and consulting domains: Investment banking: 55.2% Pass@1 (#13), +5.5 pts over GPT-5.6 Sol Management consulting: 51.6% (#16), +9.0 pts GPT-6 Sol is only 0.7 pts behind GPT-6 Astra in investment banking. These gains are not consistent across all domains. Compared with GPT-5.6, Sol’s corporate law score drops 5.7 pts to 56.2% (#27). Luna drops 2.8 pts to 55.6%. GPT-6 Luna improves most on coding work: On APEX-SWE, GPT-6 Luna scores 38.8% Pass@1, gaining +6.7 pts over GPT-5.6. Integration: 59.3%, +13.3 pts Observability: 18.3%, no change Sol scores 45.0% Pass@1 (#18), about level with GPT-5.6 Sol (45.8%). Integration: 62.3% (#13), level with GPT-6 Astra (62.0%) Observability: 27.8% (#19), down 3.7 pts For both models, debugging with production telemetry is still the hardest part. Congrats to the @OpenAI team on the launch. See full leaderboard: mercor.com/apex/apex-agents-…

Mercor has now partnered with > 300 enterprises to help them monetize their data, paying companies up to $10M per enterprise. Across these companies, we’ve encountered 90 different systems: -55% contained Slack -53% GitHub -53% Google Drive -50% Gmail -40% Figma -35% Jira A company with 200 employees runs on many of the same systems as an enterprise with 5,000+. For AI to do economically valuable work, it needs to understand how work actually happens across these systems — not just within any one of them.

APEX is now on the Specialized Intelligence Index, @FireworksAI_HQ's new leaderboard for real-work benchmarks by industry. We built APEX to test frontier models’ ability to do professional work. We’re proud to partner with Fireworks to make these benchmarks more widely available. APEX General Practitioner (MD) APEX-Agents Corporate Law APEX-SWE Find out which models lead each category. See the full results on SII: fireworks.ai/specialized-int…

Claude Opus 5.5 is the new #1 on APEX-Agents and APEX-Accounting. APEX-Agents: 73.5% Pass@1 (#1) 81.3% mean score (#1) APEX-Accounting: 15.4% Pass@1 (#1) 62.0% mean score (#1) On APEX-Agents, the new model gains +4.9 pp over Fable 5.1 (68.6%), the previous leader, and +7.6 pp over Opus 5 (65.8%). Anthropic says the biggest gains in this release are on long-running agentic tasks and knowledge work. That is what APEX-Agents measures, and the numbers agree. Opus 5.5 on APEX domains: Management Consulting: 80.0% Pass@1 (#1) Corporate Law: 71.2% Pass@1 (#3) Investment Banking: 69.3% Pass@1 (#2) Accounting: 62.0% mean score (#1) Token use can indicate domains where the model is most effective. Consulting leads at 80% and uses the fewest tokens at 2.0M per attempt. Law has the highest partial credit (85.7% mean) but costs the most at 4.9M tokens. In Accounting, Opus 5.5 gets partial credit on most tasks but fully passes only 1 in 6. More tokens can increase capability on Opus 5.5. Max effort consumes 3.50M tokens per attempt. That’s 2.1x more than Opus 5 and 1.2x Fable 5.1. But list price fell to $4/$20 per M (Opus 5 was $5/$25), and cache reads are $0.20, so 2.1x the tokens is only about 1.7x the dollars. Effort level has a significant impact on benchmark scores and token usage. Medium effort: 52.3% on 756k tokens Max effort: 73.4% on 3.50M tokens Increasing effort to max gains 21 points for 4.6x the tokens on agentic work. On APEX-Accounting, the same jump buys 2.8 points for 3.4x the tokens. When Opus 5.5 solves a task, it solves it consistently. 142 of 239 tasks passed on all 4 runs. Failing runs used 3.1M tokens vs 1.9M for passing runs, and took twice as long. 13 runs hit a hard failure, all in Corporate Law. 8 of them burned 25M to 42M tokens before dying, showing that long-horizon legal work can still send the model into a loop. Congratulations to the @claudeai team. See full leaderboard: mercor.com/apex/apex-agents-…

Don’t miss newly announced Fireworks Forge speaker: @BrendanFoody, CEO and Co-founder of @Mercor. Join the people, teams, and companies building their own frontier on open models. Nov 3, San Francisco. Apply to attend: fireworks.ai/forge#apply