#speculative-decoding — Blog

~ / blog / tag / speculative-decoding

❯ grep -r "#speculative-decoding" ~/blog

21 matches

datereadtitle
2026-10-2832m
[DGX Spark] DeepSeek V4.1 Flash on Two DGX Sparks: TensorFold TP=2 Setup, 20% Faster Single-Stream Than Qwen3.8
#dgx-spark #deepseek-v4.1-flash #tensorfold #exl3
2026-10-1619m
MiniCPM-V 4.7 35B-A3B on One RTX 2080 Ti: 111.5 tok/s with Vision, Replacing a Two-GPU 27B
#minicpm-v #2080-ti #moe #llama.cpp
2026-09-2817m
[DGX Spark] Qwen3.8-Flash-Next on two DGX Sparks: 51.9 to 72.0 tok/s, and a draft vocabulary built from my own text
#dgx-spark #qwen3.8-flash-next #vllm #mtp
2026-09-2817m
NVFP4 Without FP4 Hardware: Qwen3.8-27B on Two RTX 2080 Tis at 151.7 tok/s
#fastllm #nvfp4 #quantization #2080-ti
2026-09-2821m
Dual RTX 2080 Ti FastLLM: 16% Faster Decoding and No More Head-of-Line Blocking
#fastllm #dflash2 #speculative-decoding #2080-ti
2026-09-2124m
Two Modded RTX 2080 Tis Hit 153.8 tok/s on Qwen3.8-27B With FastLLM + DFlash2
#fastllm #dflash2 #speculative-decoding #2080-ti
2026-08-3023m
[Benchmark] GLM-5.3-Flash 320B on One DGX Spark: 15.4 tok/s with Vision Working
#dgx-spark #gb10 #glm-5.3 #llama.cpp
2026-08-2826m
[Benchmark] A 177B MoE on Three Modded 2080 Tis: 23 tok/s at 128K, 77 on a File Edit
#llama.cpp #2080-ti #moe #speculative-decoding
2026-08-2320m
[Benchmark] Qwen3.8-27B hits 65 tok/s on one DGX Spark — 12.3 without speculative decoding
#dgx-spark #gb10 #sm121 #nvfp4
2026-08-218m
[Benchmark] Free 15% Speedup on a 2080 Ti: One Broken Chat Template, One Starving MTP Head
#llama.cpp #chat-template #kv-cache #mtp
2026-07-2918m
[Junk-Tier Big Models #3] A 284B MoE on ONE 2080 Ti — and it beats my DGX Spark
#deepseek-v4-flash #moe #2080-ti #llama.cpp
2026-07-2210m
[Junk-Tier Big Models #2] Running Poolside Laguna S 2.1, a 118B Coding MoE, on ONE 22GB 2080 Ti
#laguna #poolside #moe #2080-ti
2026-07-2112m
[DeepSeek-V4-Flash] Swapping the ds4 Engine for a Free Half-Generation Speedup on One DGX Spark
#deepseek-v4-flash #dgx-spark #ds4 #speculative-decoding
2026-07-0610m
[Local LLM] Depth-1 MTP on V4-Flash: +9% on agent turns, −4% on prose — route speculative decode by workload
#dgx-spark #gb10 #deepseek-v4-flash #speculative-decoding
2026-06-2514m
[Just for Fun — Advanced] Quantizing the Draft Cache Backfired — A Counterintuitive Look at Qwen MTP (f16 ran 34% faster than q4)
#mtp #speculative-decoding #local-llm #qwen3
2026-05-2110m
Round 2 EAGLE-3 retrain didn't break the ceiling — a 60-hour null-result writeup
#gemma-4 #abliteration #eagle-3 #speculative-decoding
2026-05-1614m
EAGLE-3 fine-tune against an abliterated Gemma 4 body — Round 1 flattens the acceptance curve (plus a measurement lesson)
#gemma-4 #abliteration #eagle-3 #speculative-decoding
2026-05-146m
30 lines of docker for +34% on DGX Spark: huihui Gemma 4 FP8 + vanilla MTP n=1 deployment recipe
#gemma-4 #abliteration #mtp #speculative-decoding
2026-05-0914m
Want MTP speedup on abliterated Gemma 4? Vanilla draft can't track the modified body
#gemma-4 #abliteration #mtp #speculative-decoding
2026-05-0615m
Liftoff: Gemma 4 hits 670 tok/s aggregate on DGX Spark (108 tok/s single-stream)
#gemma-4 #mtp #speculative-decoding #vllm
2026-04-2814m
[llm-compressor] Self-Quantizing a 35B Abliterated MoE to FP8 on DGX Spark: 4 OOMs, 3 Prefix Bugs, and Why the First Success Wasn't Actually FP8
#dgx-spark #gb10 #sm121 #llm-compressor

← back to all posts