I dug up a 22GB-modded RTX 2080 Ti for ~$340 all-in (¥2079 sticker + shipping) — just enough to keep a resident 27B agent brain running on the same cheap old desktop. What the mod changes, and the gotchas.

Picking a local model, I looked at tok/s first too. Gemma 12B does 90-100 and it's great — until you put it on a kanban board, where it finishes the work and just walks away, never marking the card done. A Qwen 27B that's three times slower actually closes the loop. Why throughput is the wrong number for an agent — plus how grep almost lied to me about it.

The model card says n_ctx_train=262144. The card has 22GB. The 27B's Q4 weights are only 15.7GB. The math looks obvious: max it to 256K, plenty to spare. -c 262144, launch — loads fine, no error. A few turns of real conversation later: 503, the service restarts itself. No tidy out-of-memory in the log, just a lone 0xc0000409. nvidia-smi: free VRAM down to ~170 MiB. Where did the gigabytes go? This is the hunt: I first blamed context checkpoints, but the llama.cpp source says they live in host RAM — the real VRAM eater is the KV cache; free-VRAM-vs-context is nonlinear, and the one stable sweet spot isn't 256K — it's 128K.

Quantizing the main KV cache to q4 to save memory is fine. So I quantized the MTP draft cache too — it's just a little draft, surely a free win. It wasn't: q4 draft cache ran 29.6 tok/s, the un-quantized f16 ran 39.7, and f16 used less VRAM on top of that. The draft cache is one of the few places where quantizing is a net loss — here's the triple penalty.

On a long conversation, every message makes the model re-read the whole thing (re-prefill) before it answers — worst right after a restart or a cache eviction. Stock llama.cpp can save the KV cache to disk (--slot-save-path) but won't do it on its own — the auto-persist feature request is closed as not planned. A tiny stdlib reverse-proxy restores instead of re-prefilling: 9.9s → 1.4s on a 5K chat (7×). Mechanism, proxy design, and why I haven't shipped it yet.

I added up what my home agent pays before it reads a single word from me: ~23K tokens of overhead, and 17K of that is just the instruction manuals for its tools. Worse, it runs a hybrid model — on a cache miss it re-processes all 17K from scratch, and a single user turn can do that a dozen-plus times. This is context economics, badly underestimated. The fix isn't cutting tools; it's loading them on demand, the way skills already do.

To ease the wait on a pokey local agent, I turned on Telegram streaming — which, the way this bot did it, means rewriting the same message every fraction of a second. On a 14 tok/s brain, a single 175-second reply works out to an estimated couple hundred edit requests, which slammed into Telegram's flood control and got the whole bot benched for four minutes — final answer included. The short, ugly lesson: slow models should not fake streaming with edits. Send the finished answer once. Live logs inside.

I swapped my home agent's brain for one that decodes 30-40 tok/s instead of 14, and it felt slower. The number I'd stared at for a year — tok/s — only measures how fast tokens come out, not how long before they start. On a hybrid model, a single cache miss re-prefills the entire prompt: same box, same brain, 2.6s warm vs 216s cold. Here's the live log.

A local brain on a headless Windows box kept dying under real load: the client caught a brief 503, the service quietly restarted itself, and the application log was blank. The worst part wasn't the crash — it was that my own log truncated the one line that explained it on restart. This is the hunt to dig that reason back out from under its own log: why a self-restarting service is the best at burying the reason for its own crash, why you go to the OS-layer Event Log first, and what 0xc0000409 actually means. Full disclosure: I haven't 100% pinned the root cause — this is an open investigation, not a closed case.

The same modded 2080 Ti 22G ran Z-Image 3.4× slower on one machine than another. Not the hardware, not the OS, not a missing package — a Turing bf16→fp32 fallback hiding in one log line. One flag fixed it.

The four files are 38 GiB on disk; the card has 22. A modded 2080 Ti 22G still renders 15s of 1080p with audio in 23 minutes. Full config, measured speed and quality, then how it got there.