GPU Taskman shows per-process VRAM and an LLM weight and KV split
MIT-licensed GPU Taskman v0.3.3, released 7 November 2026, reports per-process VRAM and estimates LLM weights and KV cache.
Wildness 2/5 · Mostly tamed
Topic · 5 stories
gguf
5 stories tagged gguf on AI Tamer, newest first.
RustingStrata runs Qwen3.6-35B-A3B on a 12 GB RTX 3060 in pure Rust, splitting experts between GPU and CPU
A source-available Rust engine for Qwen GGUF models. Its author measures 113 tok/s decode vs 66.5 for tuned llama.cpp on an RTX 3060; prefill is level.
Wildness 3/5 · Partly tamedWhat a GGUF File Says Before You Load It
A GGUF file contains a map of its model, from architecture and tensor types to tokenizer details and a declared license. Here is how to read that map before use.
Wildness 2/5 · Mostly tamedQuantization labels decoded: Q4_K_M and friends
What GGUF quantization names like Q4_K_M, Q5_K_S, Q8_0 and IQ3_M mean, how size and quality trade off, and how to pick one for the memory you have.
Wildness 2/5 · Mostly tamedChoosing a local runtime for open-weight models: llama.cpp, Ollama, vLLM, and LM Studio
A practical guide to choosing a local model runtime, understanding GGUF and memory needs, and deciding when a desktop setup is enough or a server makes sense.
Wildness 4/5 · Still wild
