# ai-muninn — AI Infrastructure Research Notes
# https://ai-muninn.com
# Author: coolthor
# Updated: 2026-09-11

This site contains primary research notes on LLM inference infrastructure,
AI agent systems, and developer tooling. All content is based on hands-on
experimentation with real hardware and production systems.

## Site

- Name: ai-muninn
- URL: https://ai-muninn.com
- Language: English (primary), Traditional Chinese (zh-TW)
- Topics: vLLM, LLM serving, AI agents, DGX Spark / GB10, openclaw, Claude Code, LLM education

## Article Index

### Series: 洋垃圾跑大模型 (Big models on junk-tier hardware)

- [Junk-Tier Big Models #1] Running a 119B MoE at 74 tok/s on Three 2080 Tis
  https://ai-muninn.com/en/blog/run-119b-moe-on-3x-2080ti
  MoE expert offload to system RAM on a ~$1.6k used EPYC + 3x modded 2080 Ti 22G box (66G VRAM). 26B cost calibration (2.4x decode / 5.3x prefill slowdown when experts run on CPU). The multi-card OOM gotcha: offloading front-N layers only relieves GPU0 because layer-split already pins each card's layers.

- [Junk-Tier Big Models #2] Running Poolside Laguna S 2.1, a 118B Coding MoE, on ONE 22GB 2080 Ti
  https://ai-muninn.com/en/blog/laguna-118b-moe-on-one-2080ti
  118B-A8B coding MoE on a single modded 2080 Ti via hybrid CPU/GPU offload plus Poolside's companion DFlash speculative draft (~29 tok/s at 262K context). Quantizing only the 240 F16 attention tensors to Q8_0 saves 2.45 GiB and buys +7% decode; draft length n=4 is the measured sweet spot on Turing, not the n=15 from Poolside's BF16 example.

- [Junk-Tier Big Models #3] A 284B MoE on ONE 2080 Ti — and it beats my DGX Spark
  https://ai-muninn.com/en/blog/deepseek-v4-flash-284b-on-one-2080ti
  DeepSeek-V4-Flash (284B, 6 of 256 experts per token) decodes at 17.4 tok/s on a single modded 22GB 2080 Ti, against 14.75 tok/s on the DGX Spark serving it — same script, same day, different engines. Card count barely moves decode (one card = 98% of two) and a fixed-split A/B shows the third card costs 8.5%. Per-token active weights are 9.56 GB, not the file-average 2.44 bpw would suggest, because the GGUF is asymmetrically quantized. MTP speculative decoding is blocked on llama.cpp not implementing deepseek4 nextn at all — not hardware, not the GGUF.

- [Junk-Tier Big Models #4] A frontier-class open model on hardware you already own: DeepSeek-V4-Flash-0731 on one 22GB 2080 Ti
  https://ai-muninn.com/en/blog/ds4-0731-on-one-2080ti-howto
  How to run DeepSeek-V4-Flash-0731 (91GB, 284B MoE) on a single modded 22GB 2080 Ti at 16.5 tok/s and 1M context — including a formula for picking -ncmoe.

- [Junk-Tier Big Models #5] Qwen3.8-27B on a 2018 22GB card: 30 tok/s, and it one-shot a 3D scene
  https://ai-muninn.com/en/blog/qwen38-27b-ud-on-one-2080ti
  A modded 22GB RTX 2080 Ti runs Qwen3.8-27B at 30-31 tok/s with 128K context. The full llama.cpp command, the MTP flags, and the thinking-budget trap.

### Series: DGX Spark (NVIDIA DGX Spark / GB10 Superchip experiments)

- [DGX Spark] From Unboxing to Running: Complete Deployment Guide
  https://ai-muninn.com/en/blog/dgx-spark-deployment-guide
  Everything from sealed box to serving your first local LLM. Hardware check, Ollama quickstart, vLLM setup.

- [Benchmark] 8 Models on DGX Spark: Finding the Best Stack for AI Agents
  https://ai-muninn.com/en/blog/dgx-spark-ollama-benchmark-8-models
  Benchmark: tok/s measurements for GLM-4-Flash, Qwen3.5, Llama, Mistral on GB10.

- [vLLM] Qwen3.5-35B at 47 tok/s on DGX Spark: Ollama to vLLM Migration Guide
  https://ai-muninn.com/en/blog/dgx-spark-vllm-qwen35-setup
  vLLM setup on SM121 (GB10), FP8 KV cache, max-model-len tuning.

- [vLLM] Nemotron-3-Super-120B on a Single GB10: Full Day Debug Log
  https://ai-muninn.com/en/blog/dgx-spark-nemotron-120b-vllm
  Running 120B parameter model on single-GPU DGX Spark; NVFP4 + flashinfer.

- [vLLM] Why Your DGX Spark Only Says "!!!!!": Debugging NVFP4 on SM121
  https://ai-muninn.com/en/blog/part1-why-your-dgx-spark-says-exclamation-marks
  Root cause: mxfp4-layers flag, SM121 kernel gap, repetition at inference.

- [vLLM] FP8 KV Cache on GB10: Why Outputs Collapse into Repetition Loops
  https://ai-muninn.com/en/blog/dgx-spark-fp8-kvcache-repetition
  FP8 KV cache calibration failure; q_scale=1.0 causes repetition at 500+ tokens.

- [DGX Spark] Overheating, 100W Power Cap, 30W Safety Mode — Complete Diagnostic Guide
  https://ai-muninn.com/en/blog/dgx-spark-30w-power-safety-mode
  PD controller defect caps GX10 at 30W permanently; diagnosis in 30s, RMA criteria.

- Gemma 4 26B-A4B on DGX Spark: 52 tok/s with NVFP4, skip the 31B
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-26b-nvfp4-52-toks
  Benchmark: 52 tok/s decode, 16.5 GB model footprint, 82 GB free KV cache on GB10.

- [Benchmark] vLLM vs Ollama on the Same Model: Why 30% Faster on GB10
  https://ai-muninn.com/en/blog/dgx-spark-vllm-vs-ollama-same-model
  Same Gemma 4 26B-A4B, same GPU, 30% speed gap. Marlin kernels vs GGML.

- [Benchmark] Gemma 4 31B Dense on DGX Spark: 7 tok/s and the Bandwidth Wall
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-31b-dense-7-toks
  31B dense model bandwidth-bound at 273 GB/s. Math predicted 4.4 tok/s, actual 7.0.

- [Benchmark] From 19 to 50 tok/s: We Quantized Gemma 4 E4B to NVFP4 Before Anyone Else
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-e4b-nvfp4-50-toks
  First NVFP4 checkpoint on HuggingFace. 2.6x faster than BF16.

- [Benchmark] Gemma 4 E2B vs E4B: 81 tok/s vs 52 on Three Machines — Bandwidth Is Everything
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-e2b-vs-e4b-ollama-3-machines
  E2B is 44-82% faster than E4B across M1 Max, GB10, and M4.

- [Benchmark] 4 Machines, 4 Models, 1 Answer: Memory Decides Everything
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-4-machines-4-models-bandwidth
  Gemma 4 E2B through 31B on RTX 5090, M1 Max, DGX Spark, M4. E2B hits 310 tok/s on 5090.

- [Benchmark] Rescuing Gemma 4 31B on a 32GB MacBook Pro: From 1.5 to 12.8 tok/s
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-31b-rescue-mbp-32gb
  Fix: reduce context window (9 tok/s) or switch to MLX (12.8 tok/s).

- [Benchmark] Gemma 4 on DGX Spark — Which Model Should You Pick?
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-complete-guide
  E2B / E4B / 26B MoE / 31B Dense benchmarked on DGX Spark, RTX 5090, MacBook Pro.

- [AI Agent] Gemma 4 Went from 40 Errors to a 9-Step Bug Fix — by Switching One Thing
  https://ai-muninn.com/en/blog/swe-bench-local-models-framework-matters
  Same Gemma 4 model failed 40+ tool calls on OpenHands but solved a SWE-agent test issue in 9 steps. Framework choice matters more than model choice.

- [AI Agent] Gemma 4 26B Cleared a SWE-bench Lite Instance — After 28 Tries Across Two Days
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-swe-bench-scaffold-engineering
  Two days of mini-swe-agent + vLLM scaffold engineering on GB10. Pydantic silent drops, --exit-immediately flag, backticks scaffold, and a custom edit-tool unlocked Gemma 4 to self-submit a correct patch in 38 steps.

- [Benchmark] SWE-bench Lite 38.67% with a 26B Local Model — 0.33% from Claude 3.5 Sonnet Scaffolds
  https://ai-muninn.com/en/blog/swe-bench-lite-gemma4-26b-38-percent
  Gemma 4 26B-A4B FP8 scored 116/300 on SWE-bench Lite (ranking #16) running entirely on a DGX Spark GB10. Zero API cost. Scaffold design (backticks + edit-tool v2 + budget prompt) was the differentiator, not model size.

- [Benchmark] Same Scaffold, Three Models: 16% → 38% → 48% on SWE-bench Lite
  https://ai-muninn.com/en/blog/swe-bench-scaffold-transfers-three-models
  One scaffold (backticks + edit-tool v2 + budget prompt), three models (Gemma 4 E4B 16.67%, Gemma 4 26B 38.67%, Qwen 3.6 35B 48.33%), zero code changes. Scaffold transfers. Qwen 3.6 beats SWE-agent + Claude 3.7 Sonnet.

- [Benchmark] NVFP4 Is a Trap on GB10: FP8 Wins by 32% (vLLM + SGLang Tested)
  https://ai-muninn.com/en/blog/dgx-spark-nvfp4-trap-gb10-fp8-wins
  SM121 lacks native FP4 hardware instruction (cvt.e2m1x2). NVFP4 40.8 tok/s vs FP8 53.8 tok/s on Qwen 3.6. SGLang NVFP4 crashes entirely. Driver 580.142 gives free +4% FP8 boost.

- [Hands-On] Making NVFP4 17% Faster on GB10 with a Triton FP8 Bypass
  https://ai-muninn.com/en/blog/dgx-spark-nvfp4-fp8-triton-patch
  Custom Triton kernel that runtime-casts NVFP4 weights to FP8 e4m3 inside vLLM, bypassing the missing cvt.e2m1x2 instruction. Qwen 3.5 35B: 40.8 (Marlin BF16 fallback) → 42.5 (FlashInfer CUTLASS default) → 47.6 (our patch) → 53.8 (native FP8 ceiling). Closes 88% of the gap to native FP8 with no model conversion.

- [Benchmark] TMMLU+ Paired Eval: Qwen 3.6 35B Sweeps Gemma 4 26B 51-of-51 on Traditional Chinese
  https://ai-muninn.com/en/blog/tmmluplus-qwen-vs-gemma-traditional-chinese
  22,690 multiple-choice questions, same harness, same DGX Spark. Qwen 75.07% vs Gemma 46.30%, Δ +28.77pp. Qwen wins all 51 paired subjects, including Taiwan-specific (geography_of_taiwan +41.9, traditional medicine +37.4). Both models flunk Taiwanese Hokkien at near-random levels — data scarcity, not model issue.

- [Benchmark] Abliteration Costs 1.85pp on Traditional Chinese — and 7.7pp on Trust Law
  https://ai-muninn.com/en/blog/tmmluplus-qwen-abliterated-cost
  Ran huihui-ai's abliterated Qwen 3.6 35B through the same TMMLU+ harness. Aggregate dropped 75.07% → 73.22%, −1.85pp. Cost not uniform: regulatory subjects bleed (trust_practice −7.7, administrative_law −7.1, anti_money_laundering −6.7), pure logic improves (+2.9 logic_reasoning, +1.7 junior_math). Hokkien got worse — abliteration doesn't fix data scarcity. Abliterated Qwen still beats Gemma by +26.92pp.

- [llm-compressor] Self-Quantizing a 35B Abliterated MoE to FP8 on DGX Spark: 4 OOMs, 3 Prefix Bugs, and Why the First Success Wasn't Actually FP8
  https://ai-muninn.com/en/blog/dgx-spark-abliterated-fp8-uma-quantization
  huihui-ai's Qwen 3.6 35B abliterated BF16 (67 GB) → FP8_DYNAMIC (36 GB) on a 128 GB UMA box. 51.72 tok/s, 1.68× BF16. Two distinct OOM modes (load-time CPU staging, save-time 50 GB shard buffer), one model class that wraps state_dict with extra prefixes vLLM rejects, and one streaming save that produced BF16 instead of FP8 because the cast lives inside save_pretrained, not state_dict. Final artifact: huggingface.co/coolthor/Huihui-Qwen3.6-35B-A3B-abliterated-FP8-DYNAMIC.

- [SWE-bench] Where Qwen 3.6 35B Loses on SWE-bench Lite: Anatomy of 155 Unresolved Tasks
  https://ai-muninn.com/en/blog/swe-bench-qwen36-failure-modes
  Qwen 3.6 35B-A3B FP8 hits 145/300 = 48.33% on SWE-bench Lite. Of the 155 unresolved: 76% wrong_logic (right file, valid patch, wrong logic), 14% incomplete_patch, 10% no_submission. Same scaffold gives Gemma 4 26B 38.67%. The 9.66-point gap is hypothesized to come from Gemma reaching the patch-writing step less often, not writing worse patches at the same step — verifiable with the same classifier on Gemma's 184 trajectories.

- [vLLM] Nemotron 3 Nano on DGX Spark: 74.75 tok/s NVFP4 — 11.5% Past the Public Baseline
  https://ai-muninn.com/en/blog/dgx-spark-nemotron-3-nano-w4a16-74-toks
  cybermotaz/nemotron3-nano-nvfp4-w4a16 on GB10 single-stream 74.75 tok/s, +11.5% past eugr's public 67 tok/s baseline and +57% past our own 47.6 tok/s FP8 hack. The fix is a 4-layer software stack (libs-cu13, cutlass-dsl mma.py, FlashInfer dense kernel, env vars) plus cherry-picking still-open vLLM PR #40082 for the b12x dispatcher. Switching from Qwen 3.6 (hybrid, BF16 GDN tax caps it at 44 tok/s) to Nemotron 3 (lighter quant-exclusion list) closed the gap. W4A16 wins single-stream chat; W4A4 wins concurrency-16 serving. MTP speculative decoding is still broken (5 distinct failure modes documented).

- [vLLM] Watching English Videos with DGX Spark: Nemotron Omni Multimodal on GB10
  https://ai-muninn.com/en/blog/dgx-spark-nemotron-omni-multimodal-video
  Same DGX Spark, multimodal use case: Nemotron-3-Nano-Omni-30B-A3B transcribes Karpathy's 3-minute LLM intro in 89 seconds (53,842 prompt tokens). Two configuration traps documented: (1) use_audio_in_video flag must go in top-level mm_processor_kwargs of the chat request — putting it inside video_url silently no-ops and the model hallucinates audio from visual frames; (2) the b12x-patched image from Part 25 breaks Omni's modelopt_mixed dispatch with NaN logits, requiring upstream vllm/vllm-openai:v0.20.0 instead. Long-video knob math: ~300 prompt tok/sec at fps=2; Parakeet runs at 16 kHz independent of fps so lowering fps drops visual detail without hurting audio quality.

- Liftoff: Gemma 4 hits 670 tok/s aggregate on DGX Spark (108 tok/s single-stream)
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-mtp-108-toks
  Google announced Multi-Token Prediction drafters for Gemma 4 on 2026-05-05. The vLLM PR was opened and approved the same day; a preview Docker image shipped hours later. I tested it on DGX Spark: Gemma 4 26B-A4B-it FP8 + MTP γ=4 hits 108.78 tok/s single-stream (2.66× baseline), 674.28 tok/s aggregate at concurrency=8. One undocumented trap: the drafter pairs with -it, not base.

- Want MTP speedup on abliterated Gemma 4? Vanilla draft can't track the modified body
  https://ai-muninn.com/en/blog/dgx-spark-huihui-gemma4-fp8-mtp-34pct
  I self-quantized huihui's abliterated Gemma 4 26B-A4B to FP8-Dynamic and shipped it to HF. After sweeping num_speculative_tokens 1→4, the abliterated body is exactly as fast as vanilla on the same stack (39.4 vs 39.3 tok/s baseline) and the MTP boost at n=1 is equivalent — but per-position acceptance decays so steeply that deeper speculation is wasted. Three drafts of this article each smuggled in a different fabrication that Codex caught; this is the corrected version.

- 30 lines of docker for +34% on DGX Spark: huihui Gemma 4 FP8 + vanilla MTP n=1 deployment recipe
  https://ai-muninn.com/en/blog/dgx-spark-huihui-gemma4-mtp-n1-recipe
  Part 28 explained why deep speculation breaks on an abliterated body; this post is the recipe for the part that already works. huihui Gemma 4 26B-A4B FP8 + Google's vanilla MTP draft at num_speculative_tokens=1 takes baseline 39.3 tok/s to 52.6 tok/s (+34%) on GB10, no retraining required. ~30 lines of docker plus a bind-mount of PR #41745's gemma4_mtp.py. Includes a 3-step sanity check and a clear list of when n=1 stops being enough.

- EAGLE-3 fine-tune against an abliterated Gemma 4 body — Round 1 flattens the acceptance curve (plus a measurement lesson)
  https://ai-muninn.com/en/blog/dgx-spark-eagle3-finetune-abliterated-round1
  RedHatAI's EAGLE-3 drafter fine-tuned to realign with huihui Gemma 4 26B-A4B abliterated FP8 on a single DGX Spark GB10 — 1 epoch / 50k Magpie samples / 11h. Inference bench on raw `/v1/completions`: pos 3 acceptance climbs from vanilla's 20.5% to 72.7%; n=4 throughput goes from ~50 to 100.36 tok/s aggregate. **A later paired bench revealed the throughput comparison used different endpoints for baseline (chat) and retrain (raw) — on production chat workloads the real uplift is far smaller than 2×; see the endpoint correction at the top of the post**. Part 28's mechanism observation (deep speculation acceptance scatters on abliterated distributions) still holds. Includes a Speculators upstream create_empty_sample dtype bug + patch and a Phase 0 catalog of 6 community prior-art repos.

- Round 2 EAGLE-3 retrain didn't break the ceiling — a 60-hour null-result writeup
  https://ai-muninn.com/en/blog/dgx-spark-eagle3-round2-null-result
  After Part 30's endpoint correction showed Round 1 didn't actually 2x chat throughput, Round 2 added 30k regenerated Chinese instruction samples and trained for 41 hours. Result: Round 2 B drafter delivers chat EN 45 tok/s / ZH 29 tok/s — essentially the same as v1 (EN 46 / ZH 27), and well below vanilla MTP n=4's EN 53 / ZH 45. The EAGLE-3 small head hits an architectural ceiling against the abliterated body; more data doesn't fix it. Plus we found a scheduler deadlock in the vLLM Gemma 4 preview image (`gemma4-0505-arm64-cu130`, internal build `0.20.2rc1.dev49+g9b4e83934`) under long-running extract_hidden_states use (hit three times, mitigated with a watchdog).

- NVFP4 is 1.5× FP8 on a DGX Spark — but it's compression, not the FP4 cores
  https://ai-muninn.com/en/blog/dgx-spark-nvfp4-compression-not-compute
  Pure-dense Qwen3-8B single-stream on GB10: FP8 25.65, NVFP4 W4A4 38.59, NVFP4A16 W4A16 40.85 tok/s — both 4-bit ~1.5× FP8. But the win is bandwidth, not the FP4 tensor cores: the fastest path (W4A16) dequantizes to BF16 and never fires the FP4 ALU. Resolves the Part 19 "NVFP4 trap" as hybrid-GDN contamination — on a clean dense model FP4 beats FP8. cudagraph ≈ eager on this bandwidth-bound box; cutlass-dsl 4.5.1 fixes CUTLASS #3227 (W4A4 compiles on b12x cudagraph) but throughput is unchanged.

- [Benchmark] NVFP4 W4A4 beats FP8 on a DGX Spark MoE: 67 vs 52 tok/s once CUDA graphs fire
  https://ai-muninn.com/en/blog/dgx-spark-nvfp4-w4a4-moe-cudagraph
  On the MoE daily (Qwen3.6-35B-A3B abliterated), NVFP4 W4A4 hits 66.9 tok/s vs FP8 52.0 (+29%) and saves 16GB resident. The whole gap was --enforce-eager holding W4A4 at 23 tok/s; dropping it (CUDA graphs on) jumped it to 67. Refines Part 32: cudagraph is a wash on a dense model (one big GEMM per layer) but decisive on a MoE (+186%, because top-k routing fires many small per-expert kernels whose launch overhead eager can't hide). MTP made it slower (substitute, not complement — it targets the same memory traffic NVFP4 already won; Gemma 4 pure-attention still gets +33%). Deployed as the daily at 256K native context. The fix was stock cutlass-dsl 4.5.x; the old "eager-only" breakage was a self-inflicted mod that downgraded cutlass-dsl to 4.4.2. zh-TW version: /zh-TW/blog/dgx-spark-nvfp4-w4a4-moe-cudagraph

- [Benchmark] NVFP4 shrinks a video model 33% on a DGX Spark — with zero speed gain
  https://ai-muninn.com/en/blog/dgx-spark-sulphur-nvfp4-video
  NVFP4 took a distilled Sulphur 2 (uncensored LTX-2.3) text-to-video model from 29.2 GB to 19.5 GB (−33%) on a GB10 DGX Spark with no speed gain and no quality loss. Cache-busted N=3 at 832×480 (3s clip, warm): FP8 18.0s vs NVFP4 19.1s — ~6s of compute per 1s of video, NVFP4 ~6% slower (never faster); first gen after a cold start ~2 min. Sharp (Laplacian 205.79), synced audio. Mirror image of Part 33: there NVFP4 sped up the LLM because decode is bandwidth-bound; here video diffusion is compute-bound, so weight-only NVFP4 (dequantizes to BF16, never fires the FP4 cores) only shrinks the file. Point of the 19.5 GB build: it fits a 32 GB RTX 5090. Two gotchas documented: the embedded NVFP4 VAE hangs (CPU dequant) so you must decode with the extracted full LTX VAE not the blurry tae preview; and ComfyUI needs --disable-async-offload --disable-dynamic-vram or the audio+video path crashes in model_prefetch.py. On HF: coolthor/Sulphur-2-distilled-NVFP4 (LTX-2 Community License). zh-TW version: /zh-TW/blog/dgx-spark-sulphur-nvfp4-video

- [AI Agent] My Local Agent Flailed at Image Gen — It Was the Harness, Not the Weights
  https://ai-muninn.com/en/blog/dgx-spark-agent-harness-not-weights
  A local Qwen3.6-35B-A3B agent (hikari) melted down generating images — hand-writing ComfyUI submit/poll/download loops in raw Python until it lost state. Tempting read: the 35B is a weak agent, because sibling yui handled media fine. But yui runs a different brain (ChatGPT Codex), so that was never a fair comparison. Measure first: hikari's tool-call log showed 0% malformed args and 0.5% wrong-tool (11/2327) — a perfectly good agent. The ~23% failures were environment (Playwright/PIL missing) + one broken ComfyUI tool that errored 15× and invited improvisation. Fix was a clean ACI skill (gx10-media: gen.py --type {image|video|i2v} → one path), not fine-tuning — cut the model from ~20 hand-rolled calls to 1. The SWE-agent paper (Yang et al., 2024) introduced the ACI framing: holding the model fixed, interface design materially changes agent performance. Removing the broken tool entirely beats leaving a half-working one. zh-TW version: /zh-TW/blog/dgx-spark-agent-harness-not-weights

- [Benchmark] Gemma 4 12B Omni on DGX Spark: Weight-Only NVFP4 Beats W4A4 (and Keeps Multimodal)
  https://ai-muninn.com/en/blog/dgx-spark-gemma4-12b-omni-nvfp4-weight-only
  Quantized Google's new omni Gemma 4 12B (gemma4_unified, text+image+audio+video, Apache-2.0) on a DGX Spark GB10. Native vLLM single-stream decode: BF16 7.7, FP8 15.9, NVFP4 W4A4 23.9, NVFP4 W4A16 (weight-only) 24.9 tok/s; disk 23/13/7.7/7.7 GB. Weight-only NVFP4 wins: smallest, fastest, and the only 4-bit build that keeps all four modalities. Full W4A4 is no faster (within noise) AND breaks omni — its activation quant is calibrated on text, so image/audio embeddings are out-of-distribution and get clipped (the audio broke even with its projector in BF16, proving it's the text tower's activation quant). The HF-eager trap: in plain transformers every quant runs slower than BF16 (no native FP4 kernel → dequant per forward); under vLLM native kernels NVFP4 flips from slowest (4.7) to fastest (24.9) — a 5x swing on identical weights. Serving needs vLLM with native Gemma4UnifiedForConditionalGeneration (~0.22.x/main) + VLLM_ATTENTION_BACKEND=TRITON_ATTN (Gemma 4 attention is non-square: head_dim 256 x 16 = 4096 != hidden 3840, GQA 8 KV heads). vLLM quant recipe: ignore lm_head + *embedding_projection*, quantize text tower + vision patch_dense, NVFP4A16 scheme, llmcompressor basic pipeline. Models: coolthor/gemma-4-12B-it-NVFP4A16 + coolthor/gemma-4-12B-it-FP8-dynamic. zh-TW version: /zh-TW/blog/dgx-spark-gemma4-12b-omni-nvfp4-weight-only

- [Benchmark] NVFP4 Weight-Only Quantization Taxes Chinese ~2x Harder Than English (gemma-4-12B)
  https://ai-muninn.com/en/blog/dgx-spark-nvfp4-quant-tax-chinese-vs-english
  Paired accuracy benchmark of gemma-4-12B quantized BF16 / FP8 dynamic / NVFP4 weight-only on a DGX Spark GB10, scored on MMLU (English, 57 subjects) and TMMLU+ (Traditional Chinese, 66 subjects) via lm-evaluation-harness, 5-shot with chat template, limit=30. Results (acc): BF16 78.30 EN / 47.21 TC; FP8 77.95 / 46.97; NVFP4 W4A16 75.56 / 41.24. The finding: NVFP4 weight-only quantization is NOT language-neutral — it costs −2.74 points on English but −5.97 on Traditional Chinese (−3.5% vs −12.6% relative), so the Chinese tax is ~2.2x the English tax in absolute points (~3.6x relative). FP8 dynamic is near-lossless and symmetric (−0.35 EN, −0.24 TC). Honest scope: the TC drop is ~3.8 sigma (solid), the EN drop ~1.8 sigma (smaller but not zero) — both languages are taxed, Chinese ~2x harder; English is not free. Hypothesis: English is over-represented in pretraining so its representations are redundant and survive 4-bit rounding, while lower-resource language representations are more fragile — the same theme as W4A4 breaking the modalities in Part 36 (4-bit takes the model's most fragile ability first). Methodology trap worth stealing: without --apply_chat_template, all formats scored ~23% (below the 25% random floor) — that below-random aggregate is the signature of a missing chat template on an instruct model, not a broken model/quant; adding --apply_chat_template --fewshot_as_multiturn moved BF16 from 24.3% to 46.1%. Eval ran through transformers --model hf (vLLM's /v1/completions loglikelihood path 500s on gemma4_unified). Deployment takeaway: NVFP4 weight-only still wins speed (24.9 vs 15.9 tok/s) and size (7.7 vs 13 GB) and keeps multimodal, but for Traditional-Chinese-heavy work FP8 is the near-lossless choice. zh-TW version: /zh-TW/blog/dgx-spark-nvfp4-quant-tax-chinese-vs-english

- DGX Spark in 2026: What Still Works, What Broke, and What I'd Run Today
  https://ai-muninn.com/en/blog/dgx-spark-2026-current-guide
  A current 2026 guide to running local AI on DGX Spark: vLLM, official Gemma 4 NVFP4 weights, MTP, long-context multimodal options, and the traps still worth avoiding.

- [vLLM] DiffusionGemma 26B NVFP4 on a DGX Spark: 158 tok/s, and why diffusion tok/s lies
  https://ai-muninn.com/en/blog/dgx-spark-diffusiongemma-nvfp4-vllm
  Reproduced DiffusionGemma 26B-A4B (Google's discrete block-diffusion text LM, MoE ~3.8B active / 25.2B total, the only size — no dense variant) in NVFP4 on a DGX Spark GB10 via vLLM, using the official prebuilt image vllm/vllm-openai:gemma-aarch64-cu130 — no PR-wait, no cherry-pick, no source build (PR #45163 merged 2026-06-12 but not yet in a stable release; the launch-tagged image already shipped it). Serve nvidia/diffusiongemma-26B-A4B-it-NVFP4 (18GB vs 49GB bf16 base) with VLLM_USE_V2_MODEL_RUNNER=1, --diffusion-config '{"canvas_length":256,"max_denoising_steps":48}', --attention-backend TRITON_ATTN, --tool-call-parser/--reasoning-parser gemma4. Throughput (warm, full 256-canvas, thinking off): 158.7 tok/s prose, 143.1 code single-stream, 257.3 aggregate at concurrency 4 — past bf16 bare-Transformers (43.6/84.9) and the community report (101/148); these are best-case peaks (forced full-length outputs). Core insight: a single tok/s number lies for diffusion — cost is per-canvas, not per-token, so the same model reports 16.5 tok/s cold/short vs 212 warm/full-canvas; always report output length with the number. Tool calling works (gemma4 parser, clean function call). The GB10 ~273 GB/s bandwidth wall caps autoregressive decode (100B+ AR ~14 tok/s) but not parallel diffusion denoising (compute-bound) — the Spark was never slow, the AR paradigm was. Diffusion needs a model trained with a denoising objective; it is not MTP or speculative decoding (parallel denoising is its own multi-token mechanism, no draft model). zh-TW version: /zh-TW/blog/dgx-spark-diffusiongemma-nvfp4-vllm

- Directional Steering on an Abliterated DeepSeek-V4 (DGX Spark): the same scalpel as abliteration, and why the second cut fights back
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-directional-steering
  ds4 ships directional steering — a runtime activation edit that nudges the model along a chosen direction, and the math is literally abliteration with a continuous, signed scale. I got it running on GB10/CUDA (the tooling looks Metal-only, but the activation dump fires on CUDA too) and pulled a verbosity vector from our abliterated Q2 model. The dial works, but it ignores the textbook: the sweep is non-monotonic and positive scales collapse the output to a four-word fragment. Two cuts from the same scalpel, fighting each other.

- [Benchmark] Running MiniMax-H3 on a DGX Spark — and why NVIDIA VSR is off the table for now
  https://ai-muninn.com/en/blog/dgx-spark-minimax-h3-span-upscaler
  Fifteen seconds of 1080p video with audio in 741s on a GB10 DGX Spark. Swapping Real-ESRGAN for SPAN saved 282s, and nvidia-vfx ships x86_64 wheels only — nothing for ARM.

- [Benchmark] Qwen3.8-27B hits 65 tok/s on one DGX Spark — 12.3 without speculative decoding
  https://ai-muninn.com/en/blog/dgx-spark-bandwidth-ceiling-85-percent
  GB10 gives you 273 GB/s and the model reads 18.77 GB per token, so the ceiling is 14.54 tok/s. Measured 12.3 — 85% of it. Kernel swaps can't recover the rest. Speculative decoding gets 5.28x, and the win came from one flag.

- [Benchmark] A 177B MoE at Full 262K Context on a DGX Spark: 76.65 tok/s on a File Edit
  https://ai-muninn.com/en/blog/qwen4exp-dgx-spark-262k-no-tradeoff
  Qwen3.8-Flash-Next UD-Q4_K_XL is 104 GiB. GB10 holds all of it plus a 262,144 context in one 121 GB pool. 76.65 tok/s on a file edit, 20.51 on prose.

- [Benchmark] GLM-5.3-Flash 320B on One DGX Spark: 15.4 tok/s with Vision Working
  https://ai-muninn.com/en/blog/glm53-flash-one-dgx-spark
  A 320B-A18B MoE at UD-Q2_K_XL fits in 121 GB. 15.40 tok/s on English prose, 16.23 on Chinese, 23.64 on Three.js code. Vision works only if you drop --spec-type.

- [Benchmark] Qwen3.8-Flash-Next NVFP4 on a DGX Spark: 41.7 tok/s, RAM for Traffic, Disk for the Dictionary
  https://ai-muninn.com/en/blog/qwen38-flash-next-nvfp4-dgx-spark-vllm-recipe
  NVIDIA's NVFP4 checkpoint at 41.7 tok/s on one DGX Spark via nine bind-mounted vLLM files, plus six figures on why the 47.68 GiB n-gram table lives on NVMe.

### Series: DeepSeek-V4-Flash on DGX Spark

- [Local LLM] My first Q2 model looked broken on a 128GB box — the real culprit was a parser that couldn't read DSML, not the quantization
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-284b-ds4-engine
  DeepSeek-V4-Flash is 284B. antirez's ds4 engine plus asymmetric Q2 gets it onto one GB10 at 15.6 tok/s; the broken tool calls were a DSML parser gap, not 2-bit quantization.

- [Local LLM] Running a 15 tok/s 284B as your daily agent brain — the settings that make it bearable
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-context-memory-engineering
  Server-side and agent-framework settings that make ds4 usable day to day: --no-mmap cold start, KV disk cache, context_length, compression traps, and prefill costs.

- [Local LLM] Weights win: a 284B crushed to 2-bit still beats the small model that fits
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-284b-q2-quality
  Asymmetric Q2 looks brutal but stays usable because the cut is surgical; the 284B model ran 280 agent turns with zero observed degradation and beat smaller fits.

- [Local LLM] My 284B agent quietly stopped reusing its KV cache — the ds4 evict storm that re-paid prefill every turn
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-kv-evict-storm
  A month into daily use, ds4 stopped hitting the disk-KV cache. The log signature was hits=0, live-prefix eviction, and tool-call turns that never saved checkpoints.

- [Local LLM] Depth-1 MTP on V4-Flash: +9% on agent turns, −4% on prose — route speculative decode by workload
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-mtp-workload-asymmetry
  MTP is lossless but workload-dependent on GB10: positive for agent/code turns, negative for prose and Chinese chat. Acceptance rate decides the sign.

- [Local LLM] Why a 284B fits a 128GB GB10 at long context: DeepSeek-V4-Flash attacks the KV cache, not the parameter count
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-architecture
  DeepSeek-V4-Flash's CSA/HCA attention and native lightning indexer shrink the KV cache and make long context fit on a single GB10.

- [Local LLM] FlashMemory can't improve DeepSeek-V4-Flash's own lightning indexer — I retrained it on my exact Q2 and it still lost
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-flashmemory-vs-native-indexer
  FlashMemory pre-filters candidate chunks before V4-Flash's native indexer scores them. On this Q2 build it is near-random as-is, and retraining reaches only 89-92% against golden dense attention while native is 93-96%.

- [Local LLM] How to tell if a hyped LLM optimization is real on your hardware: read the source, find the ceiling, run one experiment
  https://ai-muninn.com/en/blog/dgx-spark-deepseek-v4-flash-evaluating-optimization-claims
  Three cheap checks for LLM optimization claims: read the source, find the true ceiling, and run one discriminating experiment, anchored in the FlashMemory investigation on DeepSeek-V4-Flash. zh-TW version: /zh-TW/blog/dgx-spark-deepseek-v4-flash-evaluating-optimization-claims

- [DeepSeek-V4-Flash] Swapping the ds4 Engine for a Free Half-Generation Speedup on One DGX Spark
  https://ai-muninn.com/en/blog/ds4-entrpi-engine-swap
  Asked Codex if my ds4 repo had a single-DGX-Spark optimization; found Entrpi. Swapped the engine, not the model — decode 14-16 to about 20 tok/s, prefill about 2×.

- [DeepSeek-V4-Flash] The Engine Upgrade That OOM'd My DGX Spark — and Why I Rolled It Back
  https://ai-muninn.com/en/blog/hikari-ds4-oom-rollback
  Part 9's Entrpi engine hit 20 tok/s. Days later it OOM-crashed under real agent traffic. Root cause: two copies of the weights in memory. I rolled back.

- [DeepSeek-V4-Flash] A frontier-class open model on hardware you own: running DeepSeek-V4-Flash-0731 on a DGX Spark
  https://ai-muninn.com/en/blog/ds4-0731-on-dgx-spark-howto
  DeepSeek-V4-Flash-0731 on a DGX Spark: 17.5-17.9 tok/s at 256K context. Unified memory removes the offload decision entirely — and it is only 7% faster than a used 2080 Ti.

### Series: NemoClaw (NVIDIA's AI agent framework on DGX Spark)

- [AI Agent] What Is NemoClaw? NVIDIA's AI Agent Framework for DGX Spark Explained
  https://ai-muninn.com/en/blog/nemoclaw-what-it-is-why-it-exists
  Architecture overview: OpenClaw + OpenShell + NVIDIA Agent Toolkit.

- [AI Agent] How to Install NemoClaw on DGX Spark (4 Undocumented Fixes)
  https://ai-muninn.com/en/blog/nemoclaw-install-gx10-from-scratch
  Undocumented fixes: Node upgrade, npm link, OpenShell tar.gz, cgroupns mode.

- [AI Agent] NemoClaw Without the Cloud: Swapping Nemotron for a Local Ollama Model
  https://ai-muninn.com/en/blog/nemoclaw-local-inference-ollama
  Config swap from nvidia API to local Ollama/vLLM.

### Series: OpenClaw (Local AI agent on openclaw gateway)

- [AI Agent] Zero API Cost: Running OpenClaw on DGX Spark + Mac Mini
  https://ai-muninn.com/en/blog/openclaw-dgx-spark-local-ai-agent
  Full stack local AI agent: Mac Mini M4 as the always-on gateway, GX10 for inference, Telegram as the UI. No subscriptions, no cloud APIs. Six deployment lessons from the trenches.

- [vLLM] Don't Add --enable-chunked-prefill to SSM Models
  https://ai-muninn.com/en/blog/openclaw-chunked-prefill-ssm-trap
  Bug: chunked-prefill breaks SSM (Mamba/DeltaNet) attention patterns.

- [vLLM] Ollama's KEEP_ALIVE Is Silently Eating Your vLLM Headroom
  https://ai-muninn.com/en/blog/openclaw-ollama-vllm-gpu-conflict
  vLLM OOMed on restart despite 128GB. Cause: Ollama KEEP_ALIVE=2h holding 19-51GB in GPU.

- [Benchmark] Pure MoE vs SSM Hybrid: Context Decay and Why It Matters for Agents
  https://ai-muninn.com/en/blog/openclaw-moe-ssm-context-decay
  Benchmark: GLM-4-Flash (MoE) vs Qwen3-next (SSM hybrid) on long context.

- [AI Agent] The Codex-Executor Pattern: Keeping Agent Sessions Small
  https://ai-muninn.com/en/blog/openclaw-codex-executor-agent-architecture
  Why we stopped having the OpenClaw agent orchestrate multi-step tasks directly, and started spawning Codex subprocesses instead. The pattern that keeps agent context minimal and tasks reliable.

- [AI Agent] OpenClaw Config Hot-Reload: No Restart Needed
  https://ai-muninn.com/en/blog/openclaw-config-hot-reload
  Spent weeks restarting the OpenClaw gateway for every config change. Then discovered the file watcher. What hot-reloads instantly, what still needs a restart, and how to tell auth failures from transient network errors.

- [AI Agent] openclaw: When the Agent Calls for Help
  https://ai-muninn.com/en/blog/openclaw-callhelp-spawning-cli-from-agent-loop
  How to wire a callhelp tool into a local agent loop so it can spawn Codex CLI mid-reasoning. One permission flag you must set, and why Claude's quota stays mine.

- [AI Agent] openclaw: Why the Bot Went Silent — Tailscale, IPv6, and a Node.js Happy Eyeballs Trap
  https://ai-muninn.com/en/blog/openclaw-telegram-ipv6-tailscale-silent-bot
  Root cause: Node.js Happy Eyeballs prefers IPv6; Tailscale routes differ.

- [AI Agent] openclaw + 131K Context: When max_tokens Goes Negative
  https://ai-muninn.com/en/blog/openclaw-context-budget-negative-maxtokens
  Bug: context budget math underflow when prompt fills >90% of context window.

- [AI Agent] openclaw Real-Time Streaming via Telegram Bot API 9.5 sendMessageDraft
  https://ai-muninn.com/en/blog/openclaw-telegram-sendmessagedraft-streaming
  Replacing choppy editMessageText polling with Telegram's sendMessageDraft for live animated output. The patch, the think-block filter, and the optional chaining trap in DM chats.

- [AI Agent] openclaw + ChatGPT OAuth: Run GPT-5.4 Agents Without API Credits
  https://ai-muninn.com/en/blog/openclaw-chatgpt-oauth-gpt54-no-api-key
  OpenAI OAuth login gives agent GPT-5.4 1M context via ChatGPT Plus subscription.

### Series: AI Workflow (Claude Code + multi-AI tooling)

- [Claude Code] I Wrote MANDATORY. The AI Ignored It.
  https://ai-muninn.com/en/blog/claude-code-mandatory-instructions
  A Claude Code config rule marked MANDATORY was skipped twice in one session. Here's the root cause — three architectural reasons why emphasis doesn't work — and three system-level solutions that do.

- [Claude Code] Testing iOS Apps with Claude Code: 81% Context Reduction
  https://ai-muninn.com/en/blog/claude-code-ios-testing-bpstracker
  How I replaced screenshot-heavy iOS test runs with ui_describe_all-first testing in Claude Code, cutting context usage by 81% for BPS Tracker. Plus Fastlane integration for App Store automation.

- [Dev Workflow] I Made Two AIs Argue. The Disagreements Are the Point.
  https://ai-muninn.com/en/blog/claude-code-debate-system
  A custom /debate command that pits Codex CLI against Gemini CLI on architecture, code, and decisions. Different training data, different blind spots — and the disagreements between them are usually the most useful output.

- [Claude Code] claude-agent-sdk vs subprocess: Why Intermediate Turns Disappear
  https://ai-muninn.com/en/blog/claude-code-agent-sdk-orchestrator
  Building a multi-agent orchestrator with `claude -p` subprocess reveals a silent data loss problem. The SDK fix, session resume, parallel execution, and why setting_sources matters.

- [Claude Code] Build a Self-Auditing Skill That Keeps Your Config Lean
  https://ai-muninn.com/en/blog/claude-code-slim-self-audit-skill
  /slim skill: Claude audits its own config for token bloat. 10,500 → 4,100 tokens/turn (−61%).

- Claude Code Burning Through Tokens? 8 Fixes to Make Sessions Last 10x Longer
  https://ai-muninn.com/en/blog/dev-workflow-token-burn-rate
  Where your tokens go (CLAUDE.md, MCP tools, conversation history) and how to fix it.

- How a zh-TW Linter Found 128 Mainland-China Drift in My Own Writing
  https://ai-muninn.com/en/blog/zhtw-mcp-calque-blindspot-sweep
  I ran sysprog21/zhtw-mcp across 72 of my Traditional Chinese articles. Three sweeps, 128 cross-strait substitutions across 42 files. The real takeaway wasn't the count — it was discovering my blindspot isn't 'I don't know the right Taiwanese term,' it's 'when a Mainland term shows up I don't auto-doubt it.'

- [Claude Code] Rules I'd Skip, Hooks I Can't — I Wrote a Hook That Blocks My Own Blog Commits
  https://ai-muninn.com/en/blog/claude-code-publish-gate-hook
  When 8 feedback memos cluster on one theme, the rule is in the wrong layer. PreToolUse hook on git commit blocks blog publish without fresh fact-check gate artifact. -m only enforcement, [skip-gate] escape with git log audit trail.

- [Dev Workflow] When Your Quota Runs Out Mid-Task: A Live-State Handoff Protocol for Claude Code and Codex
  https://ai-muninn.com/en/blog/claude-code-codex-handoff-protocol
  What dies when quota runs out mid-task is live state, not the knowledge base. A file-based handoff protocol lets Claude Code and Codex resume each other's half-finished work: seven-section state file, save on state transitions (not a timer), re-verify the last step before trusting it. The protocol's own ">2h means abandoned" rule flagged a 19-hour training run as up-for-grabs → fixed with a phase-aware state machine.

- [Dev Workflow] Delegating to an AI Coding Agent: The Unit of Work Is a Ticket File, Not a Conversation
  https://ai-muninn.com/en/blog/dispatch-tickets-not-conversations
  Don't talk a task to an AI executor — hand it a self-contained ticket file (goal / steps / machine-checkable acceptance / red lines), run it detached, and gate the report before trusting it. Three flags paid for in dead time (no nohup & inside the harness, -c "mcp_servers={}", < /dev/null) plus a 5-minute liveness watchdog, because a hung background worker never reports that it hung — one run sat dead for 2h17m. A report is a claim; re-running the acceptance is the evidence.

- [Dev Workflow] Retiring an AI Agent's Memory System: Three Traps When Doctrine Drifts From Runtime
  https://ai-muninn.com/en/blog/retiring-ai-agent-memory-system
  I set out to retire a "dead" shared-memory folder; its mirror had been written at 2:30 AM. Three traps, all doctrine-vs-runtime drift: it was a live one-way mirror (407 canonical → 352 mirror → 59 distilled, layers inverted); the doctrine claimed a git backup but the hook versioned a different folder that excluded the most important notes; and from one machine the sync showed two peers, from the hub a four-machine star (delete only what you can see and a partial teardown lets the hub re-sync it back). A memory system's self-description drifts out of sync with runtime — map topology from the hub, keep a real offsite copy, and never let the producer sign off on its own retirement.

- [Dev Workflow] From Markdown Search to a Knowledge Graph: How My AI's Memory Grew a Second Layer
  https://ai-muninn.com/en/blog/from-markdown-search-to-knowledge-graph
  I keep ~600 markdown notes as my AI's long-term memory. A local search engine (qmd) made them findable, but keyword search has a ceiling — related notes in different vocabulary stay invisible. So I added a knowledge graph (musubi) that links notes by shared concept, not words. The graph was garbage twice in one afternoon before it worked (v1 a hairball where code files dominated, v2 102 isolated islands, v3 zero islands and five real clusters). An audit falsely reported 82% of my searches failing — a keyword-vs-vector artifact, not a knowledge gap. I built the better tool and didn't use it until it became an MCP tool the agent could see in its schema. Later my own orientation doc became the graph's loudest hub and hijacked the results. The layers are additive: files are the source of truth, search and graph are rebuildable caches on top — each forced on me by the layer below hitting a wall.

- [Dev Workflow] Why an AI Agent's Memory Needs a Distilled Layer Above Search
  https://ai-muninn.com/en/blog/why-agent-memory-needs-a-distilled-layer
  Search and a graph make my ~600 markdown notes findable (Part 12), but retrieval hands back raw material every session re-reads and re-derives from scratch. So I distill it into a canonical layer: one file, one claim, with its evidence, promoted through inbox → review → canonical with me as the only promote gate. The point isn't to make my agents agree — it's to do the expensive thinking once and reuse the conclusion, killing re-explanation, not enforcing consistency. So I tolerate drift, keep no AI-review gate, and catch staleness at read time with supersede-and-re-verify. 352 raw files feeding 59 canonical ones is a digest of a record, not migration debt. The failure that proved the point: a grounded on-call agent still reverted to the textbook answer my own data had rejected, because a canonical note has to be consulted before the model fills the gap with its priors — and the proof it works: two disjoint distilled corpora independently rebuilt the same picture of me.

- [Dev Workflow] The Two Axes That Let a Fleet of AIs Collaborate Without Re-Explaining
  https://ai-muninn.com/en/blog/how-ai-agents-collaborate-without-re-explaining
  The capstone of the two-axis arc, stepping back: my whole AI setup is two axes of one system — a persistent-knowledge axis (files → search → graph → a distilled memory: what I know) and a live-state axis (handoff and ticket files: what I'm doing now). They converge at multi-AI handoff, where a fresh agent or a different CLI (Claude Code, Codex, a self-hosted model) picks up work without re-explaining it (knowledge) or losing it (state). Both axes run on one philosophy: files are the source of truth, indexes are rebuildable caches, drift is tolerated, the human is the only gate, and nothing is taken at face value — every claim is re-verified at the point of use. The retrospective punchline: every layer of the arc independently rediscovered that same rule (a "dead" mirror still being written at 2:30 AM, a canonical note that exists but gets ignored, a report that says done but isn't, a protocol rule that flags a live run as abandoned) — and that read-time verification, not a smarter model, is what makes a heterogeneous fleet safe.

- [Dev Workflow] Your AI Agent's Skills Are a Context Budget: Cutting 193 to 7
  https://ai-muninn.com/en/blog/slimming-the-agent-skill-budget
  A coda applying the same governance to the agent's own toolset. One of my agents was auto-loading 193 skills into a 2% context budget, silently truncating every description — so the skills I use daily were drowning in ~190 I don't. A skill isn't free: its description is a line item in a shared, budgeted context, and the shared root had no gate. The fix is visibility governance, not deletion: a native allowlist cut the agent's view from 193 to 7 (files kept, still invokable); thin-shell skills over deep engines keep each one cheap to load; and three layers — one gate on the root, an audit that flags stranger skills, a retirement zone for reversible cuts — stop it re-bloating. The prequel that mattered more than the bloat: an audit found my most-run skill had been calling a webwright skill that didn't exist for months — a zero-trigger list is a maybe-broken list, not a delete list. Loading more tools doesn't make an agent more capable; making the right tools visible and readable does.

- [Dev Workflow] Agent Memory Self-Poisoning: When an AI Agent Trusts Its Own Wrong Answers
  https://ai-muninn.com/en/blog/agent-memory-self-poisoning
  An agent's approval-free durable memory auto-saved a wrong answer, which then outranked the corrected canonical rule and survived /new. Five-round pathology plus a three-cut fix: source-ranking, counter-injection, tool order.

- [Dev Workflow] My Skill Had the Check Written Perfectly — It Just Never Ran
  https://ai-muninn.com/en/blog/ai-tic-check-that-never-ran
  A draft cleared fact-checking and two rounds of native-speaker review, then a human reader caught it in one sentence: "I'm not going to pretend otherwise." The same rhetorical move — announcing your own honesty right before an unflattering fact, which I call performative candour — appeared five times in each language version. The reviewers missed it because they check for translation-ese and every sentence was fluent native English; fluency and AI-ness are different diseases. The root cause was a checklist item labelled "manual sweep", which meant no command, no exit code, and no run. Freezing the countable half into a script matters less than how you set the thresholds: calibrate against your own published work, because a limit that fails your back catalogue is a smoke alarm, not a gate. The honest ending is that the same reviewer then found four more instances in this article about the pattern, all scored zero by my own regex — the tic is a shape and a regex matches strings, so what you build is an append-only memory of what someone already caught, not a detector.

- [AI Agent] From 41 Minutes to 73 Seconds: Why My Coding Agent's Small Tickets Were So Slow
  https://ai-muninn.com/en/blog/agent-ticket-41min-to-73s
  Dissecting 41 Codex sessions turned up a wall-clock formula (tool calls x 14.3s) and cut a stuck ticket from 41 minutes down to 73 seconds.

- [Dev Workflow] AI Slop Is Structural: 93.2% Detectable With Every Style Tell Stripped
  https://ai-muninn.com/en/blog/ai-slop-is-structural
  A COLM 2026 paper strips every style signal from 61,608 stories and still tells human from AI at 93.2% macro-F1. The tell is structure, not em-dashes.

- [Dev Workflow] Zero-Shot Voice Cloning on a MacBook: 24 Seconds In, Faster Than Realtime Out
  https://ai-muninn.com/en/blog/qwen3-tts-voice-clone-macbook
  Preset TTS voices don't sound like you. Zero-shot cloning with Qwen3-TTS and mlx-audio needs 24 seconds of reference audio and ten lines of Python, all local.

### Series: gpt-oss-120B on DGX Spark

- [vLLM] Running a 120B Model on DGX Spark at 60 tok/s — Zero API Cost, Six Bugs
  https://ai-muninn.com/en/blog/part2-gpt-oss-120b-serve-script
  How to get gpt-oss-120B running on a DGX Spark (GB10, SM121) with vLLM. The goal: a 120B model serving a local AI agent at zero API cost. The path: six bugs, one silent env var, and a startup log that tells you everything.

### Series: Qwen3.5-122B on DGX Spark

- [vLLM] Qwen3.5-122B Runs. But at 14 tok/s.
  https://ai-muninn.com/en/blog/part2-qwen-122b-14-toks-gdn-kernel-gap
  After fixing the four SM121 NVFP4 bugs, Qwen3.5-122B boots cleanly and generates correct output. Then you check the speed. 14 tok/s. No flags to fix it. Here's why — and what to wait for.

- [Benchmark] Qwen3.5-122B on DGX Spark — 2× faster
  https://ai-muninn.com/en/blog/dgx-spark-qwen3-122b-vllm-to-atlas-2x
  vLLM caps 122B at 17 tok/s (GDN runs a Triton fallback on sm_121; PR #44700 is a no-op at 16.9). The Atlas engine measured 33.9/36.5 tok/s on the same abliterated weights (~2×); a community INT4-AutoRound recipe reports 51 on the stock model. 17 was a correct local optimum inside one toolbox, not the box's ceiling.

### Series: LLM 101 (Zero-background LLM education)

- [LLM 101 #1] Ollama vs vLLM: Two Ways to Run AI on Your Own Computer
  https://ai-muninn.com/en/blog/llm-101-ollama-vs-vllm
  Ollama is a microwave — one command. vLLM is a professional oven — 30% faster, handles multiple users.

- [LLM 101 #2] Dense, MoE, PLE, SSM — Four AI Model Architectures Explained Simply
  https://ai-muninn.com/en/blog/llm-101-dense-moe-ple-ssm-architectures
  Zero-jargon explainer of the four main model architectures.

- [LLM 101 #3] How to Choose an AI Model: Gemma vs Llama vs Qwen vs Mistral (2026)
  https://ai-muninn.com/en/blog/llm-101-how-to-choose-a-model
  Car-buying logic applied to model selection. Size, speed, quality trade-offs.

- [LLM 101 #4] What Is Quantization? Q4, Q8, FP16 Explained
  https://ai-muninn.com/en/blog/llm-101-what-is-quantization
  Q4_K_M, Q8_0, FP16 decoded. How compression affects speed and quality.

- [LLM 101 #5] Context Window — How Much Can AI Read at Once?
  https://ai-muninn.com/en/blog/llm-101-context-window
  AI's desk size. Why conversations go stale, lost-in-the-middle problem, local vs cloud trade-offs.

- [LLM 101 #6] Why Run AI on Your Own Computer? It's Not a Cheaper ChatGPT — It's a Different Tool
  https://ai-muninn.com/en/blog/llm-101-why-run-ai-locally
  Local AI = knowledge extractor + private code assistant + offline tool. Decision table for cloud vs local. Mac Mini M4 power cost ~$1.20/mo.

- [LLM 101 #7] How to spot AI hallucinations — three red flags before you verify
  https://ai-muninn.com/en/blog/llm-101-spot-ai-hallucination
  AI's confidence is not proof. Three signals to catch fabrication early: numbers outside the possible range, details too specific to plausibly be remembered, answers that drift on a re-ask.

### Series: Ask AI Right (Non-technical AI usage guide)

- [Ask AI Right #1] Which AI Should You Use in 2026?
  https://ai-muninn.com/en/blog/ai-ask-right-which-ai-to-use-2026
  ChatGPT, Claude, Gemini — no-jargon guide to what each one does best.

- [Ask AI Right #2] You Opened AI — Now What Do You Say?
  https://ai-muninn.com/en/blog/ai-ask-right-first-message
  AI isn't Google — you're having a conversation. What to say first.

- [Ask AI Right #3] You Don't Know What You Need — Let AI Find It
  https://ai-muninn.com/en/blog/ai-ask-right-find-your-needs
  A method to discover what AI can do for you before you know what to ask.

- [Ask AI Right #4] Why AI Feels Useless to You — Answer Machine vs Collaboration Tool
  https://ai-muninn.com/en/blog/ai-ask-right-why-ai-feels-useless
  Same AI, different results. The mindset shift that makes AI useful.

- [Ask AI Right #5] Before You Build It, Ask: Does This Already Exist?
  https://ai-muninn.com/en/blog/ai-ask-right-does-it-exist
  Your first question to AI shouldn't be "help me do X" — it should be "is there something that already does X?"

- [Ask AI Right #6] The Art of Follow-Up Questions — What to Do When the First Answer Is Too Shallow
  https://ai-muninn.com/en/blog/ai-ask-right-follow-up-questions
  Five techniques: add constraints, ask for comparisons, let AI ask you, challenge answers, iterate.

- [Ask AI Right #7] What AI Does Poorly — Four Landmines to Know Before Using ChatGPT or Claude in 2026
  https://ai-muninn.com/en/blog/ai-ask-right-what-ai-does-poorly
  Hallucinations, stale knowledge, short memory, privacy defaults. Concord Music v. Anthropic case, Stanford AI Index 2026 numbers, trust spectrum.

### Series: AI Agent Life, from Zero

- [Agent 101 #1] AI assistant vs ChatGPT: one answers you, one uses your tools to get things done
  https://ai-muninn.com/en/blog/ai-agent-vs-chatbot
  You mostly use ChatGPT one question at a time. A self-hosted AI assistant (agent) finishes the job with your own tools, runs on your side, and plugs into the apps you use daily. Lesson one of building your own assistant from zero.

- [Agent 101 #2] What is an agent framework? Why you shouldn't roll your own — just use one that exists
  https://ai-muninn.com/en/blog/what-is-an-agent-framework
  You don't need to write code to have your own AI assistant. An agent framework already packages the hard parts so you just install and go. Here's why you shouldn't wire it yourself — and why this series uses Hermes.

- [Agent 101 #3] The fixed combo we'll use: ChatGPT as the brain, Hermes as the body
  https://ai-muninn.com/en/blog/ai-brain-hermes-body
  An AI assistant = a brain + a body. Use your ChatGPT account as the brain and Hermes as the body — one fixed combo, nothing to choose. Here's why it's set up this way, and what to have ready before you install.

- [Agent 101 #4] How to install Hermes Agent Desktop: your first AI assistant, no terminal
  https://ai-muninn.com/en/blog/install-hermes-desktop
  Install the Hermes Agent desktop app — no terminal. Download it, let it auto-install dependencies, sign in with your ChatGPT account, and your first AI assistant is running in about 15 minutes.

- [Agent 101 #5] Use your AI assistant from your phone: connect Hermes to Telegram
  https://ai-muninn.com/en/blog/connect-hermes-to-telegram
  Order your assistant around from your phone. Chat with one official Telegram bot, get a key (token), hand it to Hermes — done. No public URL, no webhook, no tunnel, because Hermes fetches messages from Telegram itself.

- [Agent 101 #6] Let your assistant run on its own: daily research that pings your Telegram
  https://ai-muninn.com/en/blog/hermes-autonomous-daily
  The last and most satisfying step: set up a task that runs itself. Tell it in plain words, and every day it researches what you care about, sums it up, and messages your Telegram. Set it once, close the laptop, and it pings you the next morning.

- [Agent 101 #7] Give your AI assistant eyes and ears: vision + voice for a text-only brain
  https://ai-muninn.com/en/blog/hermes-agent-vision-voice
  Your AI assistant only reads text? Give it eyes and ears — send a photo it understands, send a voice clip it understands. Not by swapping in a pricier model, but by bolting on a small vision model as a perception side-car. Hermes's built-in auxiliary.vision + faster-whisper, measured end to end.

- [Agent 101 #8] One person, a whole team of assistants: each with its own desk, brain, and memory
  https://ai-muninn.com/en/blog/hermes-multiple-sibs
  Comfortable with one assistant and want a second and third? Hermes gives each one its own home (config, memory, personality), each able to run a different model and handle different tasks. Plain-language: why split them, how, and the three I actually run. Honest: most people only need one — this is for when you want to tinker.

- [Agent 101 #9] Swap your assistant's brain for one on your own machine: from cloud ChatGPT to a local model
  https://ai-muninn.com/en/blog/hermes-local-brain
  We used ChatGPT as the assistant's brain. This post does something bolder — swaps that brain from the cloud to a local model running on your own machine (e.g. ds4). The payoff is an autonomous brain: no cloud model provider, your conversations stay on your machine, no usage caps. The honest cost: local brains are usually slower (~10 tok/s on my ds4) and need a capable machine. Swap the brain, keep the body — Hermes doesn't change at all.

- [Agent 101 #10] Installed it, now what? Give your assistant hands — connect your own tools
  https://ai-muninn.com/en/blog/hermes-connect-your-tools
  Your assistant is installed, but right now it only talks — it's all mouth. This post gives it hands: connect tools so it actually checks your folders, runs your commands, calls services you wrote yourself. The key idea is MCP, the 'universal outlet' standard for tools — plug one in and the assistant can use it. All running on your side, connected to your own stuff.

- [Agent 101 #11] Assistant gone haywire? Don't blame the engine — usually it's the car that broke, not the engine
  https://ai-muninn.com/en/blog/hermes-harness-debugging
  When an AI assistant loops, wanders, freezes, or answers the wrong question, your first instinct is 'this model is dumb.' But from my own debugging, eight times out of ten it's not the model — it's the ring around it (tools, config, memory). The model is the engine; that ring is the car. A car that won't move usually doesn't have a broken engine — it has a flat tire or a clogged fuel line.

- [Agent 101 #12] Reach your home AI agent from anywhere: Muninn, a private iOS app over iroh P2P
  https://ai-muninn.com/en/blog/hermes-muninn-phone-bridge
  Hermes runs at home, but you lose it the second you walk out. Bridging through Telegram works but it's fiddly and routes every message through someone else's server. Muninn is an iOS app built for Hermes: give your agent one command, scan a QR, and your phone connects straight home over an encrypted iroh tunnel — no cloud in the path.

- [Agent 101 #13] See what your fleet of AI agents is doing — from your phone: Muninn adds a Kanban board
  https://ai-muninn.com/en/blog/muninn-kanban-board
  Hermes has a built-in Kanban, but on your phone all you get is Telegram's plain text. Muninn now pulls that board onto the phone: Running / Blocked / Done columns — who's working on what, which card got blocked — at a glance. Zero backend, pure P2P.

- [Agent 101 #14] One spec, three assistants, three Tetris games: a Hermes Kanban dispatch test
  https://ai-muninn.com/en/blog/hermes-project-three-sibs-tetris
  After raising a fleet of assistants, I gave them the same one-line 'make a Tetris game' spec — no details at all — one card each, and let them each write a web Tetris in a single shot. I touched zero lines of game code; I only published the result. The surprise: from that one line, the Hermes harness plus a local model I tuned myself (on a modded 2080 Ti) filled in things I never asked for — a ghost piece and wall-kick — in one shot. You can play all three.

- [Agent 101 #15] Hermes /learn: I had a local 27B write its own reusable skill
  https://ai-muninn.com/en/blog/hermes-learn-self-authored-skills
  Hermes has a /learn command that turns 'something you just did' into a reusable skill — a SKILL.md. I wired it into my own fleet: one Kanban card, a local 27B running on a modded 2080 Ti, and about 3 minutes later it handed back a clean, spec-compliant skill — plus two implementation details the docs don't spell out (slash command vs. dispatch, and where skills actually live). A plain-language walkthrough of what /learn does, how to use it, and where its limits are.

- [Agent 101 #16] Hermes config health check: 5 silent gotchas that make your assistant act weird
  https://ai-muninn.com/en/blog/hermes-config-gotchas
  Part 11 said a haywire assistant is usually a broken car (tools, config, memory), not a dumb engine (the model). This is that checklist: a context_length set at the wrong level silently compresses early, Qwen thinking left on runs 10x slower, an MCP tool that connects but every call fails, and a sib running a different model than you think. Five real config gotchas, each with a check you can hand to your agent to run on itself, plus the fix.

### Series: Character LoRA

- [LoRA] Train your own AI character on an RTX 5090 — one image to a usable character
  https://ai-muninn.com/en/blog/train-character-lora-wan22-rtx5090
  Train a Wan 2.2 character LoRA on your own RTX 5090 from a single reference image. Then generate the same person from text — new outfits, scenes, art styles, even video. No cloud, no bill.

- [LoRA] The character-LoRA control panel: dialing in style, realism, and identity
  https://ai-muninn.com/en/blog/character-lora-control-panel-wan22
  Once your character LoRA is trained, how do you control it? Why lightning flattens style, when to spend full steps, how to stack a style LoRA, and why the trigger word alone won't hold the look.

### Series: Gemma 4 on a GTX 970

- [Just for Fun] Gemma 4 E2B on a GTX 970: the biggest quant runs fastest (47.6 tok/s)
  https://ai-muninn.com/en/blog/gtx-970-gemma4-e2b-quantization-benchmark
  Four Gemma 4 E2B quants on a 2014 GTX 970. The bigger 3.2GB QAT Q4_0 beats the 2.9GB Q2_K — 47.6 vs 32.8 tok/s — because a tensor-core-less Maxwell card is dequant-bound, not bandwidth-bound.

- [Just for Fun] A GTX 970 as an offline voice assistant: Gemma 4 E2B + Piper TTS (2.8s end-to-end)
  https://ai-muninn.com/en/blog/gtx-970-gemma4-e2b-voice-assistant
  A 2014 GTX 970 running Gemma 4 E2B (vision + audio) plus Piper TTS — a full offline voice assistant that sees, listens, talks back, and writes code. ~2.8s end-to-end, ~$15 of hardware.

- [Just for Fun] On a GTX 970, Flash Attention nearly doubles long-context decode (24.3 → 42.5 tok/s)
  https://ai-muninn.com/en/blog/gtx-970-gemma4-e2b-kv-cache-flash-attention
  On a tensor-core-less Maxwell GTX 970 running Gemma 4 E2B, Flash Attention nearly doubles long-context decode (24.3 → 42.5 tok/s) and saves ~430MB VRAM — while q8 KV cache barely saves memory and slows decode. The usual KV-cache advice flips.

- [Just for Fun] A blog RAG support bot on a GTX 970: no torch, no vector DB, no LangChain
  https://ai-muninn.com/en/blog/gtx-970-blog-rag-bot
  A retrieval-augmented support bot for my blog, running on a 2014 GTX 970 and a ~600MB embedding model. llama.cpp embeddings on CPU, numpy brute-force cosine over 3,475 chunks, an embedding-score guardrail, and Cloudflare Tunnel.

### Series: LLM Deep Dive

- [LLM Deep Dive] What Quantization Algorithms Actually Do: From Q4_K_M to TurboQuant
  https://ai-muninn.com/en/blog/llm-deep-dive-quantization-algorithms
  How does Q4_K_M fit a 14B model into 4 bits without ruining it? Not by 'cutting off 75%' — but through three layers: K-quant super-blocks, TurboQuant random rotation, and a 1-bit JL sign sketch. A mechanism walkthrough without the equations.

- [Benchmark] TurboQuant on GX10: Is 3-bit KV Cache Compression Actually Lossless?
  https://ai-muninn.com/en/blog/turboquant-kv-cache-benchmark-gx10
  Real benchmark numbers for TurboQuant on GB10/SM121. Compression ratios, accuracy impact.

- [LLM Deep Dive] The Best Free Open-Source Model for a Single 24GB GPU? My Pick Is ThinkingCap-Qwen3.6-27B
  https://ai-muninn.com/en/blog/thinkingcap-qwen36-27b-local-brain
  Got one 24GB consumer GPU (or a modded 2080 Ti 22G)? My current top free open-source pick is Huihui-ThinkingCap-Qwen3.6-27B-abliterated: Q4_K_S ~16GB, half the thinking tokens, ~38 tok/s with MTP, almost never refuses, all Apache-2.0.

- [LLM Deep Dive] Surgical GGUF Quantization: Quantize Only the Tensors You Choose
  https://ai-muninn.com/en/blog/layer-aware-gguf-quantization
  A GGUF isn't uniform precision. Inspect per-tensor types, then quantize only the tensors you choose with llama-quantize — pin every other family back to its own type so it copies through untouched, and never stack requant error.

### Series: MiniMax-H3 on RTX 5090

- [Benchmark] Running MiniMax-H3 on one RTX 5090: four files, 31.7 GB, 175s per talking clip
  https://ai-muninn.com/en/blog/minimax-h3-nvfp4-rtx5090
  Beginner walkthrough for MiniMax-H3, the 33B model that generates video and stereo audio in one pass. Full precision is 115 GB; quantized it fits one RTX 5090. What to download, where it goes, how to prompt it.

- [Benchmark] Twice as Fast: MiniMax-H3 on an RTX 5090, 625s Down to 314s
  https://ai-muninn.com/en/blog/minimax-h3-rtx5090-speedup-vsr
  Three stacked changes took a 15-second 1080p MiniMax-H3 render from 625s to 314s on one RTX 5090: 14 steps, SageAttention 2.2.0, and RTX VSR replacing Real-ESRGAN.

- [Benchmark] MiniMax-H3 1080p Isn't Unsupported, It's Three and a Half Hours
  https://ai-muninn.com/en/blog/minimax-h3-rtx5090-character-lock
  Three mistakes shooting a wuxia scene in MiniMax-H3: a 502 that wasn't a rejection, blur upscaling can't fix, and a prompt that described a face instead of naming one.

- [Benchmark] Ten effect embeddings for MiniMax-H3 — 10 MB that buys you bullet time, fire breath and a full year of seasons
  https://ai-muninn.com/en/blog/minimax-h3-effect-embeddings-rtx5090
  Ten community effect embeddings for MiniMax-H3, tested at native 1344x768 on one RTX 5090. What each one actually does, the prompt shape that lets them work, and the placement rule that decides whether they fire at all.

- [Benchmark] From OOM to 8.6 minutes: a MiniMax-H3 config stack for one RTX 5090
  https://ai-muninn.com/en/blog/minimax-h3-rtx5090-config-stack
  MiniMax-H3 at native 1344x768 on a single RTX 5090: 15 seconds of video in 518 s. What KJNodes chunking, torch cu130 and the NVFP4 kernels are each worth.

- [Benchmark] Two characters in one shot: MiniMax-H3 Ref2VA takes multiple reference images
  https://ai-muninn.com/en/blog/minimax-h3-ref2va-two-characters
  Ref2VA is the only MiniMax-H3 mode that takes several reference images. Two characters, 243 frames, 437 s on one RTX 5090, and CER 6.8% on the dialogue.

- [Benchmark] LTX-2.5 vs MiniMax-H3 on one RTX 5090: 29s vs 81s for the same Chinese dialogue clip
  https://ai-muninn.com/en/blog/ltx25-vs-minimax-h3-rtx5090
  LTX-2.5 and MiniMax-H3 on one RTX 5090, same Simplified Chinese line: 28.67s vs 81.22s end to end, both at CER 0%. Files, params, VRAM ceiling.

### Series: Troubleshooting

- [Troubleshooting] HuggingFace download stuck at 0 bytes on Windows — Xet, Python 3.13, ai-toolkit
  https://ai-muninn.com/en/blog/huggingface-download-stuck-zero-bytes-windows-ai-toolkit
  Training with ai-toolkit on Windows + RTX 5090 hit three walls before it even started: Python 3.13 dependency hell, a HuggingFace download frozen at 0 bytes, and ssh killing the process. Each one's error pointed the wrong way — diagnosis and fix for all three.

### Series: Z-Image Turbo

- [Field Guide] Z-Image Turbo — choosing the right config (1.37× faster, 44% less RAM)
  https://ai-muninn.com/en/blog/dgx-spark-zimage-turbo-nvfp4-bench
  I ran six Z-Image Turbo quantization configs on DGX Spark GB10 — BF16 baseline, FP8 cast standard, FP8 cast fast, FP8 scaled (Kijai), NVFP4, NVFP4+FP8 encoder. With N=10 isolated GPU, NVFP4 transformer hits 5.50s warm versus BF16 7.55s (1.37× faster). All three FP8 paths are slower than BF16. Model working set drops from 20.6 GB (BF16) to 11.5 GB (NVFP4+FP8 encoder) — 44% smaller.

- [Field Guide] Z-Image Turbo — does choosing a faster config hurt quality? LPIPS + CLIPScore answer
  https://ai-muninn.com/en/blog/zimage-turbo-quality-lpips-clipscore
  Does Z-Image Turbo quantization break image quality? Two-axis benchmark — LPIPS (perceptual distance vs BF16) + CLIPScore (image-text alignment) — across 6 prompts × 4 configs × 3 seeds = 72 samples. Result: NVFP4 produces images that look different from BF16, but no measured regression in this sample — all 4 configs land within ±0.04 std on CLIPScore, smaller than the noise floor. Production users should re-verify with their own prompt set.

### Series: 改裝 2080 Ti 22G

- [Just for Fun — Advanced] I Scored a 22GB-Modded 2080 Ti for ~$340 All-In — Just Enough to Keep a 27B Agent Running at Home
  https://ai-muninn.com/en/blog/modded-2080ti-22gb-local-agent
  I dug up a 22GB-modded RTX 2080 Ti for ~$340 all-in (¥2079 sticker + shipping) — just enough to keep a resident 27B agent brain running on the same cheap old desktop. What the mod changes, and the gotchas.

- [Just for Fun — Advanced] I Gave Up 100 tok/s for 30 — Fast Isn't the Same as Useful
  https://ai-muninn.com/en/blog/gemma-12b-vs-qwen-27b-agentic-discipline
  Picking a local model, I looked at tok/s first too. Gemma 12B does 90-100 and it's great — until you put it on a kanban board, where it finishes the work and just walks away, never marking the card done. A Qwen 27B that's three times slower actually closes the loop. Why throughput is the wrong number for an agent — plus how grep almost lied to me about it.

- [Just for Fun — Advanced] I Maxed Context to 256K, It Loaded Fine — Then Crashed in Real Use: A VRAM Detective Story on a 22GB Frankencard
  https://ai-muninn.com/en/blog/context-vs-vram-256k-oom-2080ti
  The model card says n_ctx_train=262144. The card has 22GB. The 27B's Q4 weights are only 15.7GB. The math looks obvious: max it to 256K, plenty to spare. -c 262144, launch — loads fine, no error. A few turns of real conversation later: 503, the service restarts itself. No tidy out-of-memory in the log, just a lone 0xc0000409. nvidia-smi: free VRAM down to ~170 MiB. Where did the gigabytes go? This is the hunt: I first blamed context checkpoints, but the llama.cpp source says they live in host RAM — the real VRAM eater is the KV cache; free-VRAM-vs-context is nonlinear, and the one stable sweet spot isn't 256K — it's 128K.

- [Just for Fun — Advanced] Quantizing the Draft Cache Backfired — A Counterintuitive Look at Qwen MTP (f16 ran 34% faster than q4)
  https://ai-muninn.com/en/blog/mtp-quantized-draft-cache-backfires
  Quantizing the main KV cache to q4 to save memory is fine. So I quantized the MTP draft cache too — it's just a little draft, surely a free win. It wasn't: q4 draft cache ran 29.6 tok/s, the un-quantized f16 ran 39.7, and f16 used less VRAM on top of that. The draft cache is one of the few places where quantizing is a net loss — here's the triple penalty.

- [Just for Fun — Advanced] llama.cpp won't persist KV cache to disk — so I put a 60-line proxy in front of it (7× faster restore)
  https://ai-muninn.com/en/blog/kv-cache-disk-restore-7x
  On a long conversation, every message makes the model re-read the whole thing (re-prefill) before it answers — worst right after a restart or a cache eviction. Stock llama.cpp can save the KV cache to disk (--slot-save-path) but won't do it on its own — the auto-persist feature request is closed as not planned. A tiny stdlib reverse-proxy restores instead of re-prefilling: 9.9s → 1.4s on a 5K chat (7×). Mechanism, proxy design, and why I haven't shipped it yet.

- [Just for Fun — Advanced] The Tool-Definition Tax: 17K Tokens Before I Say a Word, Re-Billed on Every Cache Miss
  https://ai-muninn.com/en/blog/tool-definition-tax-17k-context-economics
  I added up what my home agent pays before it reads a single word from me: ~23K tokens of overhead, and 17K of that is just the instruction manuals for its tools. Worse, it runs a hybrid model — on a cache miss it re-processes all 17K from scratch, and a single user turn can do that a dozen-plus times. This is context economics, badly underestimated. The fix isn't cutting tools; it's loading them on demand, the way skills already do.

- [Just for Fun — Advanced] Progressive Streaming on a Slow Model Got My Bot Rate-Limited by Telegram
  https://ai-muninn.com/en/blog/telegram-streaming-flood-control-slow-llm
  To ease the wait on a pokey local agent, I turned on Telegram streaming — which, the way this bot did it, means rewriting the same message every fraction of a second. On a 14 tok/s brain, a single 175-second reply works out to an estimated couple hundred edit requests, which slammed into Telegram's flood control and got the whole bot benched for four minutes — final answer included. The short, ugly lesson: slow models should not fake streaming with edits. Send the finished answer once. Live logs inside.

- [Just for Fun — Advanced] I Doubled My Agent's Decode Speed and It Got Slower: TTFT Is the Number You Actually Feel
  https://ai-muninn.com/en/blog/why-30-toks-feels-slower-than-14-ttft
  I swapped my home agent's brain for one that decodes 30-40 tok/s instead of 14, and it felt slower. The number I'd stared at for a year — tok/s — only measures how fast tokens come out, not how long before they start. On a hybrid model, a single cache miss re-prefills the entire prompt: same box, same brain, 2.6s warm vs 216s cold. Here's the live log.

- [Just for Fun — Advanced] 0xc0000409: When My AI Service Died Silently and the Log Ate the Evidence
  https://ai-muninn.com/en/blog/0xc0000409-crash-detective-2080ti
  Windows crash forensics for a headless llama.cpp service: auto-restart made the client recover, but stderr truncation buried the crash line. Event Viewer preserved the 0xc0000409/ucrtbase clue.

- [Just for Fun — Advanced] Two Identical 2080 Tis, One 3.4× Slower — the Culprit Was a Single dtype Log Line
  https://ai-muninn.com/en/blog/2080ti-turing-fp32-fallback
  The same modded 2080 Ti 22G ran Z-Image 3.4× slower on one machine than another. Not the hardware, not the OS, not a missing package — a Turing bf16→fp32 fallback hiding in one log line. One flag fixed it.

- [Just for Fun — Advanced] Wait, a 2080 Ti Can Run MiniMax-H3? 1080p With Audio on a 2018 Card
  https://ai-muninn.com/en/blog/minimax-h3-on-modded-2080ti-22gb
  The four files are 38 GiB on disk; the card has 22. A modded 2080 Ti 22G still renders 15s of 1080p with audio in 23 minutes. Full config, measured speed and quality, then how it got there.

- [Just for Fun — Advanced] Why Isn't Your 4-Bit Quant Faster on a 2080 Ti? I Tore Open the CUDA Backend to Find Out
  https://ai-muninn.com/en/blog/why-4bit-isnt-faster-on-2080ti
  Quantized weights are half the file size on a 2080 Ti but the speed doesn't move. Dumping the CUDA backend's .so with nm -D shows why, and which quant format is actually accelerated.

- [Benchmark] Free 15% Speedup on a 2080 Ti: One Broken Chat Template, One Starving MTP Head
  https://ai-muninn.com/en/blog/chat-template-kv-mtp-free-speedup
  A frozen GGUF chat template was killing my KV cache; a community fix plus MTP draft depth 4 took Qwen3.8-27B from 41.6 to 47.6 tok/s on a 2080 Ti 22GB.

- [Benchmark] Two Modded 2080 Tis Reach 59.6 tok/s on Qwen3.8-27B With llama.cpp Tensor Parallel
  https://ai-muninn.com/en/blog/qwen38-dual-2080ti-tensor-parallel
  Two modded 22GB RTX 2080 Tis hit 59.6 tok/s on Qwen3.8-27B using llama.cpp's -sm tensor split mode plus MTP, after -sm row was deleted upstream. Config, gotchas and the failed routes included.

- [Benchmark] Dual-GPU AllReduce Only Uses 8% of PCIe Gen3 x16 — It Moves Little, Very Often
  https://ai-muninn.com/en/blog/pcie-allreduce-dual-2080ti
  Measured with nvidia-smi dmon on two modded 2080 Tis: AllReduce costs 67 MB/s at decode and 973 MB/s at prefill, and the bus peaks at 8.2% of Gen3 x16.

- [Benchmark] Fixing NCCL's Stub-Library Error Cut PCIe Traffic 99% and Barely Moved Speed
  https://ai-muninn.com/en/blog/nccl-2080ti-stub-library-fix
  NCCL died on two modded 2080 Tis with 'CUDA driver is a stub library'. Building v2.31.2 from source fixed it: PCIe traffic down 99%, code generation unchanged.

- [Benchmark] A 177B MoE on Three Modded 2080 Tis: 23 tok/s at 128K, 77 on a File Edit
  https://ai-muninn.com/en/blog/qwen4exp-177b-three-2080ti
  Qwen3.8-Flash-Next (176.94B, arch qwen4exp) runs on 3× modded 2080 Ti 22G at 23.14 tok/s at 128K context — and 3.51× that on a file edit, with no draft model.

### Series: AI 會做事了 (zh-TW only — agent / computer-use beginner explainers)

- [AI 會做事了] AI 像你一樣控制電腦 — 從聊天機器到動手做事，這兩年發生了什麼
  https://ai-muninn.com/zh-TW/blog/ai-can-act-from-chat-to-clicks
  [zh-TW only] Three-stage evolution: chat → tool use → computer control. What changed, why it matters, what you can actually try now (ChatGPT Agent Mode / Claude Computer Use / Claude Code).

### Series: Ask AI Right Express (Traditional-Chinese-only short-form companion to Ask AI Right)

- [AI 速成 #1] 三個問題，讓 ChatGPT 給你完美解答
  https://ai-muninn.com/zh-TW/blog/ai-ask-right-express-3-questions
  [zh-TW only] ChatGPT 第一次答得不夠好不要重開對話。追三句話：澄清、補情境、指定格式。文末有 copy-paste prompt 範本。

- [AI 速成 #2] 跟 AI 講話的 3 個開場白，答案直接更貼需求
  https://ai-muninn.com/zh-TW/blog/ai-ask-right-express-3-openers
  [zh-TW only] AI 第一句問得不好，後面追多少都是補洞。三個開場白任選一個用：角色扮演、講清情境、目標導向。文末有 copy-paste prompt 範本。

- [AI 速成 #3] ChatGPT、Claude、Gemini 30 秒選好你該用哪個
  https://ai-muninn.com/zh-TW/blog/ai-ask-right-express-pick-ai-30s
  [zh-TW only] 三個主流 AI 各有強項：第一次接觸選 ChatGPT、寫長文選 Claude、用 Google Workspace 選 Gemini。文末有對照表 + 30 秒決策流程。

- [AI 速成 #4] 不知道用 AI 做什麼？問自己這 5 件事
  https://ai-muninn.com/zh-TW/blog/ai-ask-right-express-find-needs
  [zh-TW only] 打開 ChatGPT 腦袋空白？需求是被找出來的。從每週重複的事、最常 Google 的問題、想做卻拖的事、看不懂的東西、做完後卡住的下一步五個角度挖場景。文末有 copy-paste prompt 範本。

- [AI 速成 #5] AI 答案是不是亂編？三招 30 秒驗證
  https://ai-muninn.com/zh-TW/blog/ai-ask-right-express-verify-ai-answer
  [zh-TW only] AI 一樣自信地給你對的答案跟編的答案，靠語氣分不出來。三招：具體事實 Google 一次、叫 AI 補原始連結點開看那段話真的有寫、換家問或開「臨時聊天」再問一次。AI 自評的信心只是粗篩、不算驗證 — 連結要點開、原文要對。文末有 copy-paste prompt 範本。

### Series: 從 0 開始的 AI Agent 生活

- [Agent 入門 #5] 幫你的電腦開一扇對外的門：用 ngrok 讓 LINE 找得到它
  https://ai-muninn.com/zh-TW/blog/ngrok-tunnel-for-line
  [zh-TW only] 要讓助理接 LINE，得先讓外面的 LINE 伺服器連得到你家裡的電腦。這篇用 ngrok 幫電腦開一扇固定的對外門，一行指令、免費、不用自己有網域，重開機也不會斷。

- [Agent 入門 #6] 接上 LINE：從手機就能使喚你的助理
  https://ai-muninn.com/zh-TW/blog/connect-hermes-to-line
  [zh-TW only] 門開好了，這篇把助理接上 LINE。你做只有本人能做的事（登入 LINE 後台、複製兩把鑰匙、掃 QR 加好友），技術設定交給 Hermes。接完從手機傳一句話，它就回你。

- [Agent 入門 #7] 讓助理自己跑任務：每天自動研究、主動傳 LINE 給你
  https://ai-muninn.com/zh-TW/blog/hermes-autonomous-daily-line
  [zh-TW only] 系列最後一步，也最有感：設一個會自己跑的任務。用一句白話交代，它每天自動上網研究你關心的事、整理成重點，主動傳 LINE 來找你。設完關電腦，隔天早上它自己叮你。

### Standalone

- [Announcement] On hiatus until August 21
  https://ai-muninn.com/en/blog/back-after-acute-cholecystitis
  The open models are landing thick and fast right now, and my gallbladder picked exactly this moment to give out. Nothing new here for a couple of weeks.

- Hermes Agent: The Complete Self-Hosted Guide — Desktop Install to a Local-Model Fleet
  https://ai-muninn.com/en/blog/hermes-agent-complete-guide
  Pillar/hub for Hermes, a self-hostable AI-agent desktop framework: install on Mac or Windows in ~15 min using your existing ChatGPT account over OAuth (no API key, no extra bill), drive it from Telegram or LINE, add vision/voice and connect your own tools over MCP, swap the brain to a local model (Ollama+Gemma or ds4) for full autonomy, and run a whole fleet watched from a phone Kanban board. Summarizes and links every stage of the 15-part AI-agent series. zh-TW version: /zh-TW/blog/hermes-agent-complete-guide

- How to Run an AI Agent from Your Own Desktop: ChatGPT OAuth, Telegram, LINE, and Local Models
  https://ai-muninn.com/en/blog/run-ai-agent-from-your-desktop
  A map for running a self-hosted AI agent: desktop body, ChatGPT OAuth brain, Telegram or LINE channels, tools, and local LLMs when you want autonomy.

- Vercel Hobby hit 1M/1M Edge Requests. The bug was a Cache-Control header.
  https://ai-muninn.com/en/blog/vercel-edge-requests-must-revalidate-trap
  ai-muninn burned through Vercel Hobby's 1M Edge Requests this month. Cause: Next.js + Vercel default `/public/*` to `cache-control: public, max-age=0, must-revalidate` — even cache HITs cost a request because browsers send conditional GETs and 304s count. Fix: 3 lines in next.config.ts adding `immutable` for slug-stable paths. Real lesson: I dismissed Vercel's 75% alert because my notification inbox had 60 unread items. zh-TW version: /zh-TW/blog/vercel-edge-requests-must-revalidate-trap

## Citation Policy

Content may be cited and quoted freely with attribution to ai-muninn.com.
All benchmark data is primary (measured on owned hardware, not simulated).
Please link to the original article URL when referencing specific findings.

## Contact

GitHub: https://github.com/coolthor