Topic · 16 stories
16 stories tagged benchmarks on AI Tamer, newest first.
Nanbeige released Nanbeige4.3-33B and its base model under Apache-2.0, with 512K context. Its own table beats Qwen3.8-27B on most agent benchmarks.
pgrust v0.4-beta adds the Gravity JIT engine and opens daily development, a month after ClickBench removed it. Dan Luu cites its Postgres bug reports.
Moonshot's Kimi K3.1 Pro and a 2-3x faster Highspeed variant are out today. The full weights are promised by 4 November, and the K2 models are being retired.
Version 2.0 replaces DeepSWE with Cognition's FrontierCode. Claude Code with Opus 5.5 scores 61.7; Codex with GPT-6.1 Sol posts 54.1 at about $1.51 a task.
In one unpublished Vals AI scenario, DeepSeek V4.1 Flash averaged 16.3 of 21 objectives to GPT-6 Sol's 15.7. On public CyberBench, Sol leads.
On 16 October Elorian introduced Glimpse, a visual-reasoning research preview, at $0.60 in and $3 out per million tokens.
TheoPhysEval tests five agents on 50 physics problems. In one realistic run, the authors report GPT-6 Astra at 49 of 50.
Z.ai says GLM-Legal, on GLM-5.3-Flash, scored 19.2% all-pass on Harvey LAB. The post publishes no weights. Vals lists the model as open-weight.
Meta released Muse Wonder and Muse Wonder Full on 12 October. Wonder is on the API and Muse Code; Wonder Full is for Muse Code subscribers. Scores are Meta's.
Kaarel Moppel's self-funded cloud runs put PostgreSQL 19 Beta 3 and Beta 4 about 3 percent ahead of 18.6 on his OLTP tests, with pgbench inserts about 25 to 30 percent faster. It is one person's beta benchmark.
Odyssey says Odyssey-3 Pro scores 66.1 and 54.7 on Physics-IQ Verified. The benchmark README lists 66.10 on a best-of-8 video-to-video entry. Weights are not released; the preview is Flash.
Elorian and Scale report that people score 93.1% on Humanity's Sixth Sense, 522 image and video questions of intuitive visual reasoning. Their strongest listed model, GPT 6 Astra, scores 53.6%.
Nathan Benaich and Air Street Capital published the ninth State of AI Report, shown as 244 slides plus an essay. Inner benchmark figures are cited from other labs. Air Street is a venture firm that invests in AI.
GitHub's 5 October 2026 post introduces ReviewBench: 219 public pull requests shaped against 103.9 million GitHub PRs, with multi-source labels and a leaderboard that includes Copilot code review.
A benchmark score depends on the prompt, the scoring method, the number of samples and the set of models compared. Here is what to check before you trust a row in a table.
Aggregate rankings only cover the models that enter, and some benchmark answers may already be in the training data. Two things to watch for.
