dab-data-agent-benchmark

Evaluate data agents on realistic multi-database queries across 54 tasks spanning 12 datasets, 9 domains, and 4 DBMS systems. Reveals frontier models achieve only 38% pass@1 accuracy, with 85% of failures from incorrect planning rather than data selection. Benchmark captures critical challenges: heterogeneous database integration, identifier reconciliation,…

Independent GitHub snapshot · Automated classification · Original documentation governs