Measure quantization damage with KL divergence alongside perplexity
Save full-precision logits once with llama.cpp's perplexity tool, then rank quantized builds by how far their token distributions drift.
Wildness 1/5 · Tamed
Topic · 6 stories
quantization
6 stories tagged quantization on AI Tamer, newest first.
Weaviate 1.40 ships namespaces as generally available, if you have a license
Weaviate 1.40.0 marks namespaces generally available. A merged pull request says namespaced endpoints return 403 without a license. The notes do not name a plan.
Wildness 2/5 · Mostly tamedWhat a GGUF File Says Before You Load It
A GGUF file contains a map of its model, from architecture and tensor types to tokenizer details and a declared license. Here is how to read that map before use.
Wildness 2/5 · Mostly tamedQuantization labels decoded: Q4_K_M and friends
What GGUF quantization names like Q4_K_M, Q5_K_S, Q8_0 and IQ3_M mean, how size and quality trade off, and how to pick one for the memory you have.
Wildness 2/5 · Mostly tamedVectors, Plainly · Part 2 of 6
Vectors, Plainly, part 2: inside the index, from HNSW to hybrid search
How HNSW graphs, IVF cells and product quantization make nearest-neighbour search fast, and how filters, keyword fusion and reranking change what a query returns.
Wildness 2/5 · Mostly tamedQuantization, plainly: what 16-bit, 8-bit and 4-bit models trade
A practical guide to model weight precision, memory estimates, common quantization formats, quality checks and choosing a quant for your workload.
Wildness 3/5 · Partly tamed
