A new analysis of the US Benchmark Map examines how well existing AI benchmarks measure capability across the American labor force. The study finds that 90% of US working hours involve tasks not adequately covered by unsaturated, public, benchmarks—tests designed to measure AI agent performance on real-world work.
This gap suggests a mismatch: while AI labs benchmark narrowly defined tasks, the broader economy relies on workers doing complex, overlapping work that current tests don't capture. The finding raises questions about whether benchmark results accurately reflect AI readiness for deployment across different industries and job categories.
This is an AI-written summary. The original reporting belongs to mercor.com. How we write these · Spot a mistake?