Mercor's benchmark map: why it says 90% of US work hours aren't covered by AI benchmarks, and what the number measures

Mercor's benchmark map, published in mid-October 2026, puts a number on a familiar complaint about AI evaluation: "90% of American working hours are spent on tasks with little or no coverage from existing benchmarks." The AI training-data company mapped 29 benchmarks onto the US government's catalog of 18,797 job tasks and found that only 208 of 923 occupations have even one task with meaningful coverage. That matters for every claim about how good frontier models are at "real work." The headline number rests on a low bar for "covered," an LLM judge, and a benchmark list Mercor chose. Mercor also has a commercial stake in the result.

This story is based on Mercor's benchmark map page and its Benchmarks and Methodology tabs, the research papers it cites, and TechCrunch's reporting on Mercor's business, checked October 18, 2026. Mercor's page carries no publication date; Mercor announced it on X on October 15, 2026 (UTC).

The short version

• The claim: 90% of US working hours go to tasks with "little or no" benchmark coverage.
• Coverage: 208 of 923 occupations, and 924 of 18,797 tasks, are covered.
• What "covered" means: an LLM judge matched the task to at least 5 prompts across 29 benchmarks.
• Skew: tasks rated exposed to AI are about 6 times more likely to be covered, and higher-paid occupations are overrepresented.
• Mercor's caveat: the results are "provisional" and probably understate coverage.
• Mercor's interest: it sells expert data and evaluations to AI labs and makes the APEX benchmarks used in the study.

How did Mercor calculate the 90% figure?

Mercor combined three ingredients:

1. Tasks. It used O*NET, the US Department of Labor's occupation database: 923 occupations in 22 job families, broken into 18,797 tasks.
2. Coverage. An LLM judge tagged each prompt in 29 benchmarks with the O*NET tasks it corresponds to, following a method from Wang et al., a March 2026 paper from Carnegie Mellon and Stanford researchers. Mercor's Methodology tab names the judge as GPT 5.6 Luna. A task counts as covered "if our LLM judge has mapped it to at least 5 prompts from our 29 benchmarks."
3. Hours. To turn task coverage into a share of working hours, Mercor used "LLM-derived estimates of the working time share of each ONET task" from Hatgis-Kessell et al. That June 2026 paper estimates how long each task takes by asking language models to compare tasks pairwise, then combines that with how often ONET says a task occurs. For wage-bill comparisons, Mercor used Bureau of Labor Statistics mean wages and employment, assuming a 40-hour week.

So the 90% is a model-based estimate built on top of two other model-based estimates: LLM tagging of benchmark prompts and LLM-derived task durations.

Which benchmarks are included?

Twenty-nine benchmarks that Mercor selected for public availability (at least 25 public tasks), agentic tasks, "economic realism," economic diversity (with a cap per job family) and not yet being saturated. Mercor lists them on the map's Benchmarks tab (grouped here by the job families Mercor tags them with):

• Software, computing and science: SWE-bench Verified, SWE-Lancer, Terminal-Bench 2.1, ALE-Bench, SciCode and The Agent Company.
• Office, sales and finance work: tau2-Bench Telecom, tau3-Banking, CRMArena-Pro, EnterpriseOps-Gym, SpreadsheetBench 2 and AutomationBench (Zapier).
• Law and professional services: Harvey LAB and Scale AI's PRBench.
• Engineering, maintenance and design: AssetOpsBench, PHMForge, AEC-Bench, CadEval and Graphic-Design-Bench.
• Health, care and teaching: PhysicianBench, CARE-Bench and TeachArena.
• Cross-occupational: OpenAI's GDPval, $OneMillion-Bench and Agents' Last Exam.
• Mercor's own: APEX-1, APEX-Agents, APEX-Accounting and APEX-SWE.

Four of the 29 are Mercor's. The public-availability rule has an exception for them: Mercor says it includes its APEX benchmarks because it has access to their held-out prompts, and its index lists APEX-Accounting with 10 public tasks, below the 25-task bar. The APEX benchmarks are also the only ones the page analyzes in detail.

That selection shapes the answer. Mercor acknowledges that capping benchmarks per job family "likely understates" coverage in heavily benchmarked fields such as software engineering, and that leaving out saturated benchmarks may understate coverage of easier tasks.

Which jobs are covered, and which aren't?

By Mercor's count, Computer & Math has the highest share of covered occupations (29 of 36), followed by Community Service and Legal, and Mercor says benchmarks are "full of tasks in Software Engineering and Financial Analysis." Lawyers' tasks are heavily covered, while legal secretaries and court clerks barely are.

Last verified: October 18, 2026 (Mercor benchmark map; selected rows)

Six of the 22 job families have no covered occupations at all. Mercor also flags physical work: citing an NBER paper by Dingel and Neiman, which estimates that only 37% of US jobs can be done entirely from home, it says about 63% of jobs involve physical tasks "which would be hard to incorporate into current benchmarks."

Mercor also found that tasks rated as exposed to AI, using the 2023 labels from Eloundou et al., are about 6 times more likely to be covered, and that the tasks O*NET rates most important within an occupation are about 6 to 10 times more likely to be covered than the least important. Tasks from occupations with bigger wage bills are overrepresented even after controlling for exposure.

Does "not covered" mean AI can't do the work?

No. The map measures where benchmarks exist, not what AI can do. Exposure and coverage are different things, as our explainer on the IMF's 40% figure explains for exposure. And a task "covered" by 5 prompts isn't necessarily measured well. The 5-prompt threshold shows whether a task appears in benchmarks at all, not how well it is measured, so "covered well," as some headlines put it, overstates what the map counts.

Mercor lists further limits itself:

• Low statistical power: a few low-quality benchmark items can bias scores.
• No quality weighting yet for task difficulty or benchmark quality.
• Subjective criteria, such as "economic realism," in choosing benchmarks.
• Automation only: benchmarks measure whether AI completes tasks, not how it helps people do them.
• Validation gap: the LLM tagging was checked against human experts only on three APEX benchmarks. Reviewers checked only the judge's first-choice task and accepted it in 76.7% of 1,763 judgments, while the map counts every task the judge listed (3.1 per prompt on average). APEX-SWE was tagged after the review.

The broader finding isn't new. Wang et al. mapped 43 agent benchmarks and 72,342 tasks onto 1,016 occupations in March 2026 and found the same tilt toward software and coding. Mercor's contribution is converting coverage into working hours and wage bills.

What does Mercor get out of this?

Mercor isn't a neutral observer. It connects AI labs with domain experts such as doctors and lawyers who produce training data, and charges fees on their work, TechCrunch reported in October 2025. TechCrunch reported in July 2026 that CEO Brendan Foody said Mercor's annualized revenue run rate had passed $2 billion. Mercor also builds and publishes the APEX benchmarks and leaderboards.

The map's conclusion, that most of the economy lacks good evaluations, describes a market for exactly the expert-built evaluations Mercor sells. APEX is the only benchmark family the page analyzes in detail. Its pilot weights APEX scores by ONET task for four occupations; for investment banking, it reports Claude Opus 5.5 Max at 73.3% with equal prompt weighting and 70.2% with equal task weighting, and says fewer than a third of relevant ONET tasks have enough APEX coverage to include. The page includes no conflict-of-interest disclosure. That doesn't make the analysis wrong, and Mercor is open about its methods and limits, but readers should weigh it accordingly.

What it means for builders

If you're choosing a model for work outside software, finance or law, public benchmarks probably say little about your tasks, and Mercor's map is a reasonable prompt to build a small evaluation set from your own work. Our guide to coding-agent evaluations explains why even well-covered areas need care, and Epoch AI's InnovationEval findings show how agents' own write-ups can overstate results. Treat the 90% as a directional estimate from an interested party, not a precise measurement.

What's confirmed, reported and uncertain

• Confirmed (Mercor's pages): the 90% claim, the 208-of-923 and 924-of-18,797 counts, the 5-prompt definition, the names of the 29 benchmarks, the method and expert-review figures, the stated limits, and the APEX pilot figures.
• Confirmed (cited papers): the Hatgis-Kessell et al. time-share method and the Wang et al. benchmark-mapping study.
• Reported (TechCrunch): Mercor's business model and revenue run rate.
• Uncertain: how sensitive the 90% is to the choice of benchmarks, how well the LLM tagging works outside APEX, and how the hours aggregation weights each occupation in detail.