Topic · 7 stories
7 stories tagged long-context on AI Tamer, newest first.
TokenGate, from Stanford and Berkeley researchers, trains a gate that drops unneeded context tokens early, cutting prefill time and memory on small models.
A ByteDance Seed preprint says token offset can swing DeepSeek-V4 retrieval. The 40-point gap is the base checkpoint. Post-trained gaps are smaller.
StepFun's Step 5 Preview, a 600B-parameter MoE with 1M-token context, reached OpenRouter and Vercel AI Gateway on 8 October. StepFun says open weights follow on 15 October; until then it is an API model.
Long prompts can fill GPU memory through the key-value cache. Moving that cache to the CPU saves GPU space, with a possible speed cost.
On 3 October 2026 Aleph Alpha released Kolibri, a bilingual German-English mixture-of-experts model with 78.1B total parameters, 3.46B active, Apache 2.0 weights, and context up to 1M tokens.
NaiveAI’s Naive-N0.5-Flash (Sep 27, 2026): open-weight 309B MoE / 15.5B active, native 1M hybrid SWA–DSA, MIT on HF. API ($0.10/$0.40/$0.01) and NaiveRT remain future-tense—NaiveRT GitHub still 404 at the time of writing (promised by Oct 12). Benches/tok/s are NaiveAI harness claims only.
The context window is a working memory, not a perfect one — accuracy fades as it fills, and "just give it more text" quietly stops working.
