#quantization — Blog

~ / blog / tag / quantization

❯ grep -r "#quantization" ~/blog

28 matches

datereadtitle
2026-09-2817m
NVFP4 Without FP4 Hardware: Qwen3.8-27B on Two RTX 2080 Tis at 151.7 tok/s
#fastllm #nvfp4 #quantization #2080-ti
2026-09-2124m
Ternary Bonsai 2 on One RTX 2080 Ti: Qwen3.8-27B in 7.2 GB Scores 130 of 140
#ternary-bonsai #qwen3.8 #2080-ti #llama.cpp
2026-09-209m
[LLM Deep Dive] A quantization lab: what quantization is, how it works, where the error comes from
#quantization #int4 #llm #tooling
2026-08-2826m
[Benchmark] A 177B MoE on Three Modded 2080 Tis: 23 tok/s at 128K, 77 on a File Edit
#llama.cpp #2080-ti #moe #speculative-decoding
2026-08-2011m
[Just for Fun — Advanced] Why Isn't Your 4-Bit Quant Faster on a 2080 Ti? I Tore Open the CUDA Backend to Find Out
#2080-ti #turing #quantization #gguf
2026-08-0614m
[Just for Fun — Advanced] Wait, a 2080 Ti Can Run MiniMax-H3? 1080p With Audio on a 2018 Card
#2080-ti #turing #minimax-h3 #comfyui
2026-08-0421m
[Benchmark] Running MiniMax-H3 on one RTX 5090: four files, 31.7 GB, 175s per talking clip
#minimax-h3 #nvfp4 #rtx-5090 #comfyui
2026-07-2313m
[LLM Deep Dive] Surgical GGUF Quantization: Quantize Only the Tensors You Choose
#quantization #gguf #llama.cpp #llama-quantize
2026-07-2210m
[Junk-Tier Big Models #2] Running Poolside Laguna S 2.1, a 118B Coding MoE, on ONE 22GB 2080 Ti
#laguna #poolside #moe #2080-ti
2026-07-2112m
[Junk-Tier Big Models #1] Running a 119B MoE at 74 tok/s on Three 2080 Tis
#llama.cpp #moe #expert-offload #2080-ti
2026-07-1811m
[Just for Fun — Advanced] Two Identical 2080 Tis, One 3.4× Slower — the Culprit Was a Single dtype Log Line
#2080-ti #turing #comfyui #z-image
2026-06-127m
[Local LLM] Weights win: a 284B crushed to 2-bit still beats the small model that fits
#dgx-spark #gb10 #deepseek-v4-flash #quantization
2026-06-1210m
[Local LLM] My first Q2 model looked broken on a 128GB box — the real culprit was a parser that couldn't read DSML, not the quantization
#dgx-spark #gb10 #deepseek-v4-flash #ds4
2026-06-0910m
[Just for Fun] Gemma 4 E2B on a GTX 970: the biggest quant runs fastest (47.6 tok/s)
#gemma-4 #quantization #gtx-970 #llama.cpp
2026-06-059m
[Benchmark] NVFP4 Weight-Only Quantization Taxes Chinese ~2x Harder Than English (gemma-4-12B)
#dgx-spark #gb10 #gemma-4 #nvfp4
2026-06-046m
[Benchmark] Gemma 4 12B Omni on DGX Spark: Weight-Only NVFP4 Beats W4A4 (and Keeps Multimodal)
#dgx-spark #gb10 #gemma-4 #nvfp4
2026-06-019m
[Benchmark] NVFP4 shrinks a video model 33% on a DGX Spark — with zero speed gain
#nvfp4 #sulphur-2 #ltx-2.3 #dgx-spark
2026-05-309m
NVFP4 is 1.5× FP8 on a DGX Spark — but it's compression, not the FP4 cores
#nvfp4 #fp8 #dgx-spark #gb10
2026-05-0415m
[Field Guide] Z-Image Turbo — does choosing a faster config hurt quality? LPIPS + CLIPScore answer
#z-image #comfyui #nvfp4 #fp8
2026-05-0412m
[Field Guide] Z-Image Turbo — choosing the right config (1.37× faster, 44% less RAM)
#z-image #comfyui #nvfp4 #fp8
2026-04-2814m
[llm-compressor] Self-Quantizing a 35B Abliterated MoE to FP8 on DGX Spark: 4 OOMs, 3 Prefix Bugs, and Why the First Success Wasn't Actually FP8
#dgx-spark #gb10 #sm121 #llm-compressor
2026-04-2214m
[Hands-On] Making NVFP4 17% Faster on GB10 with a Triton FP8 Bypass
#nvfp4 #fp8 #triton #dgx-spark
2026-04-218m
[Benchmark] NVFP4 Is a Trap on GB10: FP8 Wins by 32% (vLLM + SGLang Tested)
#nvfp4 #fp8 #dgx-spark #gb10
2026-04-1511m
[LLM Deep Dive] What Quantization Algorithms Actually Do: From Q4_K_M to TurboQuant
#llm #quantization #k-quant #turboquant
2026-04-1012m
[LLM 101 #4] What Is Quantization? Q4, Q8, FP16 Explained
#llm #quantization #ollama #beginner
2026-04-079m
[Benchmark] From 19 to 50 tok/s: We Quantized Gemma 4 E4B to NVFP4 Before Anyone Else
#gemma-4 #e4b #nvfp4 #fp8
2026-03-308m
[Benchmark] TurboQuant on GX10: Is 3-bit KV Cache Compression Actually Lossless?
#turboquant #kv-cache #quantization #vllm
2026-03-216m
[vLLM] FP8 KV Cache on GB10: Why Outputs Collapse into Repetition Loops
#vllm #fp8 #kv-cache #gb10

← back to all posts