Topic · 5 stories
5 stories tagged attention on AI Tamer, newest first.
Grouped-query attention gives several query heads one key-value head pair. That shrinks decoding caches, with speed and quality gains that depend on the model and workload.
RoPE rotates query and key coordinate pairs so their attention score depends on relative position. The math helps explain context extension and its limits.
A model’s weights are only part of its memory needs. The attention cache grows with the conversation, and saving space can slow generation.
A simple test for whether an AI assistant respects my attention: what happens when I ask it to stop.
The context window is a working memory, not a perfect one — accuracy fades as it fills, and "just give it more text" quietly stops working.
