#llama.cpp — Blog

~ / blog / tag / llama.cpp

❯ grep -r "#llama.cpp" ~/blog

27 matches

datereadtitle
2026-08-3023m
[Benchmark] GLM-5.3-Flash 320B on One DGX Spark: 15.4 tok/s with Vision Working
#dgx-spark #gb10 #glm-5.3 #llama.cpp
2026-08-2816m
[Benchmark] A 177B MoE at Full 262K Context on a DGX Spark: 76.65 tok/s on a File Edit
#dgx-spark #gb10 #llama.cpp #moe
2026-08-2826m
[Benchmark] A 177B MoE on Three Modded 2080 Tis: 23 tok/s at 128K, 77 on a File Edit
#llama.cpp #2080-ti #moe #speculative-decoding
2026-08-2320m
[Benchmark] Fixing NCCL's Stub-Library Error Cut PCIe Traffic 99% and Barely Moved Speed
#nccl #llama.cpp #tensor-parallel #2080-ti
2026-08-2323m
[Benchmark] Dual-GPU AllReduce Only Uses 8% of PCIe Gen3 x16 — It Moves Little, Very Often
#llama.cpp #tensor-parallel #pcie #nvlink
2026-08-2215m
[Benchmark] Two Modded 2080 Tis Reach 59.6 tok/s on Qwen3.8-27B With llama.cpp Tensor Parallel
#llama.cpp #tensor-parallel #2080-ti #qwen3.8
2026-08-218m
[Benchmark] Free 15% Speedup on a 2080 Ti: One Broken Chat Template, One Starving MTP Head
#llama.cpp #chat-template #kv-cache #mtp
2026-08-187m
[Junk-Tier Big Models #5] Qwen3.8-27B on a 2018 22GB card: 30 tok/s, and it one-shot a 3D scene
#qwen3.8 #unsloth #2080-ti #llama.cpp
2026-08-0212m
[DeepSeek-V4-Flash] A frontier-class open model on hardware you own: running DeepSeek-V4-Flash-0731 on a DGX Spark
#deepseek-v4-flash #0731 #dgx-spark #gb10
2026-08-0220m
[Junk-Tier Big Models #4] A frontier-class open model on hardware you already own: DeepSeek-V4-Flash-0731 on one 22GB 2080 Ti
#deepseek-v4-flash #0731 #moe #2080-ti
2026-07-2918m
[Junk-Tier Big Models #3] A 284B MoE on ONE 2080 Ti — and it beats my DGX Spark
#deepseek-v4-flash #moe #2080-ti #llama.cpp
2026-07-2313m
[LLM Deep Dive] Surgical GGUF Quantization: Quantize Only the Tensors You Choose
#quantization #gguf #llama.cpp #llama-quantize
2026-07-2212m
[LLM Deep Dive] The Best Free Open-Source Model for a Single 24GB GPU? My Pick Is ThinkingCap-Qwen3.6-27B
#qwen3.6 #thinkingcap #abliterated #local-llm
2026-07-2210m
[Junk-Tier Big Models #2] Running Poolside Laguna S 2.1, a 118B Coding MoE, on ONE 22GB 2080 Ti
#laguna #poolside #moe #2080-ti
2026-07-2112m
[Junk-Tier Big Models #1] Running a 119B MoE at 74 tok/s on Three 2080 Tis
#llama.cpp #moe #expert-offload #2080-ti
2026-07-1012m
[Just for Fun — Advanced] 0xc0000409: When My AI Service Died Silently and the Log Ate the Evidence
#local-llm #llama.cpp #windows #crash-forensics
2026-07-0314m
[Just for Fun — Advanced] I Doubled My Agent's Decode Speed and It Got Slower: TTFT Is the Number You Actually Feel
#local-llm #ai-agent #ttft #llama.cpp
2026-07-0211m
[Just for Fun — Advanced] Progressive Streaming on a Slow Model Got My Bot Rate-Limited by Telegram
#local-llm #ai-agent #telegram #streaming
2026-06-2714m
[Just for Fun — Advanced] The Tool-Definition Tax: 17K Tokens Before I Say a Word, Re-Billed on Every Cache Miss
#local-llm #ai-agent #context #llama.cpp
2026-06-2611m
[Just for Fun — Advanced] llama.cpp won't persist KV cache to disk — so I put a 60-line proxy in front of it (7× faster restore)
#local-llm #llama.cpp #kv-cache #ttft
2026-06-2514m
[Just for Fun — Advanced] Quantizing the Draft Cache Backfired — A Counterintuitive Look at Qwen MTP (f16 ran 34% faster than q4)
#mtp #speculative-decoding #local-llm #qwen3
2026-06-2416m
[Just for Fun — Advanced] I Maxed Context to 256K, It Loaded Fine — Then Crashed in Real Use: A VRAM Detective Story on a 22GB Frankencard
#local-llm #llama.cpp #qwen3 #vram
2026-06-2316m
[Just for Fun — Advanced] I Gave Up 100 tok/s for 30 — Fast Isn't the Same as Useful
#local-llm #ai-agent #qwen3 #gemma
2026-06-2212m
[Just for Fun — Advanced] I Scored a 22GB-Modded 2080 Ti for ~$340 All-In — Just Enough to Keep a 27B Agent Running at Home
#rtx-2080-ti #gpu-mod #local-llm #ai-agent
2026-06-1414m
[Just for Fun] A blog RAG support bot on a GTX 970: no torch, no vector DB, no LangChain
#gemma-4 #gtx-970 #rag #llama.cpp
2026-06-1410m
[Just for Fun] On a GTX 970, Flash Attention nearly doubles long-context decode (24.3 → 42.5 tok/s)
#gemma-4 #gtx-970 #flash-attention #kv-cache
2026-06-0910m
[Just for Fun] Gemma 4 E2B on a GTX 970: the biggest quant runs fastest (47.6 tok/s)
#gemma-4 #quantization #gtx-970 #llama.cpp

← back to all posts