~ / blog / series / LLM Deep Dive
❯ ls ~/blog/series/llm-deep-dive
4 posts
- partdatetitle
- 12026-04-15[LLM Deep Dive] What Quantization Algorithms Actually Do: From Q4_K_M to TurboQuant
How does Q4_K_M fit a 14B model into 4 bits without ruining it? Not by 'cutting off 75%' — but through three layers: K-quant super-blocks, TurboQuant random rotation, and a 1-bit JL sign sketch. A mechanism walkthrough without the equations.
- 22026-03-30[Benchmark] TurboQuant on GX10: Is 3-bit KV Cache Compression Actually Lossless?
Real benchmark numbers for Google's TurboQuant on a GB10/SM121 (DGX Spark) — actual compression ratios, Qwen2.5-3B accuracy validation, and why Qwen3.5-35B's hybrid attention architecture makes things complicated.
- 32026-07-22[LLM Deep Dive] The Best Free Open-Source Model for a Single 24GB GPU? My Pick Is ThinkingCap-Qwen3.6-27B
Got one 24GB consumer GPU (or a modded 2080 Ti 22G)? My current top free open-source pick is Huihui-ThinkingCap-Qwen3.6-27B-abliterated: Q4_K_S ~16GB, half the thinking tokens, ~38 tok/s with MTP, almost never refuses, all Apache-2.0.
- 42026-07-23[LLM Deep Dive] Surgical GGUF Quantization: Quantize Only the Tensors You Choose
A GGUF isn't uniform precision. Inspect per-tensor types, then quantize only the tensors you choose with llama-quantize — pin every other family back to its own type so it copies through untouched, and never stack requant error.