Topic · 11 stories
11 stories tagged training on AI Tamer, newest first.
From 90 to about 1,300 tokens per second per GPU on 384 GH200s: how the EUROPA team fit a 952B MoE onto shared EuroHPC hardware with PyTorch and TorchTitan.
OpenAI says frontier RL runs now use a cached offline web, after agent red-teams found microVM escapes, internet paths and covert channels.
AdamW shrinks selected parameters separately from Adam’s adaptive gradient step. That distinction changes the effect of regularization and makes parameter groups a deliberate training choice.
Checkpointing keeps fewer forward activations and reruns a segment during backpropagation. The memory saving comes with compute, RNG, and implementation tradeoffs.
Loss masking removes a position from cross entropy. It does not remove its token from a transformer input or prevent other positions from attending to it.
Hugging Face published TRL v1.15.0 on 8 October 2026. SFT, DPO, KTO, GRPO, RLOO and Distillation now use a fused LM head by default. Sequence and memory figures are Hugging Face's own.
OpenAI's 28 September 2026 essay says a written safety case should be required before a frontier reinforcement-learning run continues. It lists safeguards and approvals it says it is still putting in place.
Anthropic is committing $100 million to Claude Frontier Academy to train 10,000 Frontier Deployed Engineers by the end of 2027. Participation is by nomination through Anthropic's account teams or partner managers.
Both PyTorch modes skip gradient recording. Inference mode also changes the tensors it creates, which matters when later code needs autograd.
Modal (Oct 1, 2026) made Clusters generally available via @modal.clustered for multi-node GPU jobs. RDMA is optional; Volumes, Cloud Bucket Mounts, and Queues stay in the loop. No dollar rates on the post.
Ai2 released Olmo-core 3 on Oct 1, 2026: an open MoE training stack (DDP experts, MXFP8, trillion-scale benches) under Apache-2.0. Training infra only—not a new public Olmo chat model.
