Topic · 10 stories
10 stories tagged mixture-of-experts on AI Tamer, newest first.
Baidu released a 280B-total, 14B-active pre-training checkpoint on Hugging Face. Its custom license bars use in the EU, UK and South Korea.
A source-available Rust engine for Qwen GGUF models. Its author measures 113 tok/s decode vs 66.5 for tuned llama.cpp on an RTX 3060; prefill is level.
Microsoft's 15 October post says MAI-Base-2 pre-training is finished. Business Insider reports a staff remark that it beat the model behind GPT-5.6 Sol.
Shanghai AI Laboratory's site calls Atria Dawn an official release. Full weights are marked coming soon, and its pages split on 256K and 512K context.
On 15 October Meta released Muse Spark Open weights: about 816B total, 42B active, gated on Hugging Face under the Muse Model License.
Sonos published tract 0.23.9 on 15 October with a tract_moe_ffn operator. The deploy runtime is marked WIP. crates.io still lists 0.21.19 as newest.
StepFun's official Step 5 Preview weights are on Hugging Face, including BF16 and NVFP4 repos, under StepFun's own license.
Liquid AI released LFM2.5-24B-A2B on 13 October, a 24B on-device model it says uses 2.45B active parameters. Commercial use is license-limited.
A sparse mixture-of-experts router can choose an expert that has no remaining token slots. Here is what overflow means in Switch Transformer and how capacity changes the tradeoff.
Reflection's 5 October 2026 post introduces Beam, a text-only 501B mixture-of-experts model with 23B active. The blog says effective context is 1M tokens. The API docs list a 256K window.
