~ / blog / series / LLM Deep Dive
❯ ls ~/blog/series/llm-deep-dive
6 posts
- partdatetitle
- 12026-04-15[LLM Deep Dive] What Quantization Algorithms Actually Do: From Q4_K_M to TurboQuant
How does Q4_K_M fit a 14B model into 4 bits without ruining it? Not by 'cutting off 75%' — but through three layers: K-quant super-blocks, TurboQuant random rotation, and a 1-bit JL sign sketch. A mechanism walkthrough without the equations.
- 22026-03-30[Benchmark] TurboQuant on GX10: Is 3-bit KV Cache Compression Actually Lossless?
Real benchmark numbers for Google's TurboQuant on a GB10/SM121 (DGX Spark) — actual compression ratios, Qwen2.5-3B accuracy validation, and why Qwen3.5-35B's hybrid attention architecture makes things complicated.
- 32026-07-22[LLM Deep Dive] The Best Free Open-Source Model for a Single 24GB GPU? My Pick Is ThinkingCap-Qwen3.6-27B
Got one 24GB consumer GPU (or a modded 2080 Ti 22G)? My current top free open-source pick is Huihui-ThinkingCap-Qwen3.6-27B-abliterated: Q4_K_S ~16GB, half the thinking tokens, ~38 tok/s with MTP, almost never refuses, all Apache-2.0.
- 42026-07-23[LLM Deep Dive] Surgical GGUF Quantization: Quantize Only the Tensors You Choose
A GGUF isn't uniform precision. Inspect per-tensor types, then quantize only the tensors you choose with llama-quantize — pin every other family back to its own type so it copies through untouched, and never stack requant error.
- 52026-09-16[LLM Deep Dive] What actually happens inside one transformer layer — four real models, four data flows
A cube you pull apart into layers. What model.ai-muninn.com shows for Qwen3-30B-A3B, Qwen3-8B, Qwen3.8-Flash-Next and DeepSeek-V4.1-Flash, and how to read it.
- 62026-09-20[LLM Deep Dive] A quantization lab: what quantization is, how it works, where the error comes from
A clickable INT4 / FP8 / FP4 lab: one outlier drops two of four weights to zero, and the scale itself is quantized, so the group maximum is off by 4.77e-7.