Surge's DAYJOB: Tax benchmark: the best AI agent passes about one tax job in ten

Surge AI’s new DAYJOB: Tax benchmark asks AI agents to do what a tax professional does: take a client request with incomplete instructions and messy records, and deliver a return, workpaper, review or advice letter.

The test

There are 50 environments across eight areas, from individual and small-business tax to partnerships, estates, international tax and disputes. They contain 1,789 files, a median of 30 per assignment and up to 123: mostly PDFs, plus spreadsheets, Word files, emails, images and CSVs. One task includes a photo of a handwritten receipt; another has a dispatch log showing crews working in five states for a client that thinks it only operates in Alabama.

An attempt passes only if it meets at least 95% of the task’s grading criteria, which cover calculations, evidence, the issues that matter and the finished deliverable. Tax experts wrote the tasks, and each went through a three-layer expert review.

Results

Claude Sonnet 5.5 leads with a 10.8% pass rate, ahead of Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%. The full leaderboard is on Surge’s benchmark page.

What goes wrong

Surge’s examples are about judgment, not arithmetic. In one, a client asks whether to sell her condo in October 2026 or January 2027. Both dates qualify for the home-sale exclusion, but eligibility runs out around the end of August 2027. In Surge’s five attempts each, GPT-6 Astra never mentioned that deadline, while Opus 5.5 caught it every time.

In another, a separating couple asks how to split retirement accounts. Opus 5.5 assumed in all five attempts that they were not legally married, although their records pointed to a possible informal marriage in Texas. Astra flagged the need to check their marital status in four of five.

Why it matters

Most tax benchmarks hand the model a defined question. This one checks whether an agent notices the question the client did not ask, which is where real advice goes wrong. A dataset is on Hugging Face; Surge asks people to email it for access to the full benchmark.