#mtp — Blog — ai-muninn

~ / blog / tag / mtp

❯ grep -r "#mtp" ~/blog

16 matches

datereadtitle
2026-09-2124m
Ternary Bonsai 2 on One RTX 2080 Ti: Qwen3.8-27B in 7.2 GB Scores 130 of 140
#ternary-bonsai #qwen3.8 #2080-ti #llama.cpp
2026-09-0623m
[Benchmark] Qwen3.8-Flash-Next NVFP4 on a DGX Spark: 41.7 tok/s, RAM for Traffic, Disk for the Dictionary
#dgx-spark #gb10 #vllm #nvfp4
2026-08-2320m
[Benchmark] Qwen3.8-27B hits 65 tok/s on one DGX Spark — 12.3 without speculative decoding
#dgx-spark #gb10 #sm121 #nvfp4
2026-08-2215m
[Benchmark] Two Modded 2080 Tis Reach 59.6 tok/s on Qwen3.8-27B With llama.cpp Tensor Parallel
#llama.cpp #tensor-parallel #2080-ti #qwen3.8
2026-08-218m
[Benchmark] Free 15% Speedup on a 2080 Ti: One Broken Chat Template, One Starving MTP Head
#llama.cpp #chat-template #kv-cache #mtp
2026-08-187m
[Junk-Tier Big Models #5] Qwen3.8-27B on a 2018 22GB card: 30 tok/s, and it one-shot a 3D scene
#qwen3.8 #unsloth #2080-ti #llama.cpp
2026-07-2918m
[Junk-Tier Big Models #3] A 284B MoE on ONE 2080 Ti — and it beats my DGX Spark
#deepseek-v4-flash #moe #2080-ti #llama.cpp
2026-07-2212m
[LLM Deep Dive] The Best Free Open-Source Model for a Single 24GB GPU? My Pick Is ThinkingCap-Qwen3.6-27B
#qwen3.6 #thinkingcap #abliterated #local-llm
2026-07-078m
DGX Spark in 2026: What Still Works, What Broke, and What I'd Run Today
#dgx-spark #gb10 #gemma-4 #vllm
2026-07-0610m
[Local LLM] Depth-1 MTP on V4-Flash: +9% on agent turns, −4% on prose — route speculative decode by workload
#dgx-spark #gb10 #deepseek-v4-flash #speculative-decoding
2026-06-2514m
[Just for Fun — Advanced] Quantizing the Draft Cache Backfired — A Counterintuitive Look at Qwen MTP (f16 ran 34% faster than q4)
#mtp #speculative-decoding #local-llm #qwen3
2026-06-0110m
[Benchmark] NVFP4 W4A4 beats FP8 on a DGX Spark MoE: 67 vs 52 tok/s once CUDA graphs fire
#nvfp4 #w4a4 #fp8 #dgx-spark
2026-05-146m
30 lines of docker for +34% on DGX Spark: huihui Gemma 4 FP8 + vanilla MTP n=1 deployment recipe
#gemma-4 #abliteration #mtp #speculative-decoding
2026-05-0914m
Want MTP speedup on abliterated Gemma 4? Vanilla draft can't track the modified body
#gemma-4 #abliteration #mtp #speculative-decoding
2026-05-0615m
Liftoff: Gemma 4 hits 670 tok/s aggregate on DGX Spark (108 tok/s single-stream)
#gemma-4 #mtp #speculative-decoding #vllm
2026-04-2814m
[llm-compressor] Self-Quantizing a 35B Abliterated MoE to FP8 on DGX Spark: 4 OOMs, 3 Prefix Bugs, and Why the First Success Wasn't Actually FP8
#dgx-spark #gb10 #sm121 #llm-compressor

← back to all posts