[DeepSeek-V4-Flash] The Engine Upgrade That OOM'd My DGX Spark — and Why I Rolled It Back

The short version: the fast engine keeps a second copy of the weights

Here's the plain-English version for a friend who doesn't do this for a living. My little AI box holds one big model in memory — think of it as a thick reference book the machine reads from to answer questions. The book almost fills the shelf. Last week I switched to a faster program for reading it, and it was faster. But it turned out that program gets its speed by making a second, reorganized copy of the entire book and keeping both on the shelf at once. Two copies barely fit. It runs fine as long as the questions stay short — but the faster program made me greedy, so I started feeding it much longer conversations. The moment I did, there was no room left for the scratch space a long question needs, and the whole thing crashed. The speed was real. It just quietly ate the room my longer questions needed, and I didn't notice until I asked for one.

I bragged in Part 9 — then I got greedy with context

Part 9 was a victory lap. I swapped the stock antirez/ds4 engine for Entrpi, a Blackwell CUDA fork, kept my abliterated model untouched, and got decode from 14-16 to about 20 tok/s and prefill roughly doubled. Same brain, faster body, zero model change. I ended that post telling you not to leave a free speedup on the table.

The model is "Hikari" (光羽) — my resident DeepSeek-V4-Flash brain for a home agent fleet, running on one DGX Spark (GB10, 128GB unified memory, about 119 GiB usable). It's the thing my agents actually talk to all day.

Then the speed made me greedy. A faster engine meant I could give my agents more context — longer conversations, more history to keep in the window — so I cranked it up. And that's when Hikari started falling over: some turns completed fine; others hung for two minutes, and the whole thing felt slower than it had before the upgrade. My first guess was the model itself: maybe the thinking traces had grown too long. Should I turn thinking off? No. Thinking was never the issue. I was chasing the wrong thing because I hadn't yet looked at the one number that mattered. The free speedup from Part 9 had a price tag stapled to the back, and it was time to read it.

Goal: find the real cause, then keep the fast engine or roll it back

I wanted two things, in order. First, the actual mechanism — not "it feels slow" but the specific reason, measured. Second, a decision I could defend: keep Entrpi and its 20 tok/s and live with the crashes, or roll back to the old engine, eat the slower decode, and get stability. No swapping models — Hikari's abliterated weights stay exactly as they are, same as Part 9. Only the engine is on the table.

1.4 tok/s: the server was waiting, not computing

The tell was in the server's own numbers. Entrpi reports its per-chat timings, so I stopped guessing and read decode_tok_s. On a slow turn it said 1.4 tok/s. Normal is about 20. That's not "a bit off" — it's 14× slower, the difference between a paragraph in a few seconds and a paragraph in a minute.

Then came the fingerprint that told me what kind of slow it was: GPU utilization sat at 5% and the CPU was idle, yet it was still crawling at 1.4 tok/s. A model that's computing hard runs the GPU hot. A model doing 1.4 tok/s at 5% GPU isn't computing — it's waiting. The working set had been paged out, and every token was faulting weight pages back into memory before it could do the math. /v1/models answered instantly (that's just metadata), but /chat timed out (that has to touch the weights). The engine wasn't thinking slowly; it was starved for memory and thrashing.

So the real problem was never speed. It was memory. Something was pushing the box over the edge. The next two things I tried both looked like the fix and both were detours.

Detour 1: dropping context to dodge managed KV just moved the OOM

At 512K context, Entrpi's boot log warns: using managed KV cache for ctx=524288 ... may degrade performance. The warning plus the thrashing pointed to an obvious theory: managed KV was the slow path, get off it. A restart, by itself, brought decode back to 20.4 tok/s, which "confirmed" the theory in the worst way: it made me think the managed-KV path was the disease and a fresh start was the cure.

So I dropped context from 512K to 320K, which uses the native KV path and dodges the managed warning entirely. Clean fix, on paper. Except 320K then OOM'd — the systemd journal was blunt about it: killed by the OOM killer. I hadn't cured anything. I'd just traded a slow path for a dead one. The memory pressure was still there; lowering context only changed which straw broke the camel's back.

Detour 2: removing --warm-weights fixed daily crashes but not big prompts

Second theory, and a better one. The --warm-weights flag forces all the weights resident at boot. Drop it, and maybe the OS can breathe. So I removed it — and my daily traffic stopped crashing. Ordinary requests with a 31K-token prefill, which had been falling over, now ran clean. That felt like the answer.

It wasn't. I sent a genuinely big prompt — 211K tokens — and it OOM'd anyway. Removing the flag had raised the ceiling, not removed it. There was still a wall; I'd just moved it far enough back that I stopped hitting it on ordinary days. Two detours in, I had a workaround and no cause.

The turning point: my OLD engine used the same flags and never OOM'd

The break came from the user, not the logs. I was framing this as a flags-and-context tuning problem when the observation that mattered was one sentence: "but my old ds4 version never OOM'd."

That reframed everything. The old engine ran the same --warm-weights and the same 256K context — and had never once been killed. Same flags, same context, same model, same box. The only variable that changed between "never crashes" and "crashes under load" was the engine binary itself. It was never the flags. It was Entrpi.

Root cause: Entrpi keeps two copies of the weights

Here's what Entrpi does to get decode from 16 to 20. At startup it repacks the weights into a Blackwell CUDA layout optimized for its kernels — a 78.71 GiB device-artifacts copy. That's the copy the runtime actually computes from. The problem: the original 80.76 GiB mmap of the GGUF is still resident at the same time. Two full copies of the model, side by side, in the same 119 GiB pool.

Do the arithmetic. 78.71 + 80.76 is about 159 GiB of weights wanting to live in 119 GiB of usable memory. It "works" only because a chunk of the mmap can be paged, which leaves the box running with roughly 2 GiB of real headroom. Two gigs.

Then it clicked: Entrpi's README sizes the box for exactly this — 119 GiB usable is "enough for the model + ~250 MB KV @ 16k context." The two copies fill it right up to that ~16K line, with almost nothing to spare. Which is fine, until you get greedy the way I did and crank the context to 320K or 512K — 20-30× past what the box was sized for. Two gigs can't hold the context I bolted on, so a big prefill's activation spike tips it past 119 GiB and the OOM killer fires. The old ds4-186-524 engine never repacks: one copy of the weights, about 11 GiB of headroom, room to absorb a big prefill without falling over.

So the +4 tok/s from Part 9 wasn't free after all. It was bought with the exact headroom my bigger contexts needed. And to be fair to Entrpi, none of this is hidden: the README draws that 16K line in plain sight, and the repo has a handful of issues and zero PRs with nobody reporting this OOM. I don't read that as a maintainer hiding a bug — I read it as me running an engine tuned for 16K contexts at 20-30× that. It's a documented tradeoff I walked into, not a fix waiting to land.

Why not just lower the context or the KV precision? (KV is a red herring)

The instinct is to blame the KV cache and shrink it. It doesn't help, and it's worth saying why so you don't waste the afternoon I nearly did. DeepSeek-V4-Flash's KV is tiny: 256K context is about 5.9 GiB. Entrpi already defaults to FP8 primary KV plus FP4 indexer compression — there is no lower precision knob to turn. Dropping from 512K to 256K saves roughly 5.4 GiB of KV (512K ≈ 11.3 GiB, 256K ≈ 5.9 GiB) — but the box is over-committed by about 40 GiB of duplicated weights, so trimming KV barely moves the needle; a big prefill still OOMs. The thing filling the 119 GiB is the 78.71 GiB of repacked weight artifacts, not the KV. You cannot tune your way out of a second copy of the model.

The decision: I rolled back to the old engine

I put Hikari back on the old ds4-186-524 engine at 256K, and measured it properly rather than trusting the vibe.

	Entrpi (fast)	Old ds4-186-524 (stable)
Decode	20 tok/s	15.5 (15.57 / 15.51)
Prefill	761 tok/s	324 tok/s
Weight copies in memory	2 (repack + mmap)	1 (mmap)
Headroom	~2 GiB	~11 GiB
Big prefill	OOM at 211K	stable through a 68K-token prefill
Restarts under real traffic	many (OOM-reload cycles)	0

Read the bottom two rows, because that's the whole decision. Entrpi is genuinely faster on both decode and prefill. It also OOMs, and every OOM triggers a reload. That's the part I'd been misreading as "the model is slow."

Here's the insight I didn't have going in: the slowness I felt was never the 4.5 tok/s decode gap between the two engines. Nobody feels 15.5 vs 20 on a chat turn. What I felt was the OOM-reload cycle — a crash, then systemd's Restart=on-failure relaunching the server, then 80-plus seconds to reload the weights before it could answer again. Two minutes of dead air, several times a day, dressed up as "the engine feels sluggish." The old engine, at a lower headline tok/s, delivers every turn without ever going dark. For daily agent traffic, stability beats the benchmark number. It's not close.

Debug: forensics on a box with no live log

This last section is the gnarly part — how I actually pinned this down on a box that had lost its logs. Skip it unless you like this kind of thing; the decision above stands without it.

① Verify the runtime, don't trust reports — and be ready for the logs to be gone. The running server's stdout was piped to a dead tmux socket — /proc/<pid>/fd/1 -> socket:[…] pointing at a session that no longer existed. No live log to tail. I had to reconstruct "which engine is this, and where is it stuck" from three sources that survive a lost terminal: the systemctl --user journal, the server's own per-chat timings (decode_tok_s / ttft_ms), and /proc/<pid>/exe for the actual binary plus its cmdline flag fingerprint. That last one bit me for a minute — I briefly mistook Entrpi's ds4-server for the old binary, because both executables are named ds4-server. The only way to tell them apart is the flags: --warm-weights / --power / --kv-disk-dir mark the one, their absence the other.

② The systemd unit is --user scope, not system. systemctl status ds4entrpi-8000 returned "could not be found," and I nearly wrote down "there's no supervisor, it won't come back on its own." Wrong scope. It was a --user unit the whole time — active, enabled, Restart=on-failure. If you check the wrong scope you'll draw exactly the wrong conclusion about whether the service self-heals.

③ What --warm-weights does — and why I call the fix an A/B, not a proof. The flag maps to model_warm_weights (ds4.c:1915), which calls MADV_WILLNEED and then touches every page, forcing all 80 GiB of the weight mmap resident at boot. The runtime doesn't need that — it serves from the repacked device artifacts (the expert weights served from device artifacts boot line) — so a fully-resident mmap is 80 GiB of pressure the box can't spare. I removed the flag and my daily 31K-token crashes stopped. I'm reporting that as an empirical A/B, not a source-level proof: the flag definitely faults 80 GiB in at boot, but the exact reclaim behavior under pressure is fiddly (clean file-backed pages stay reclaimable either way, and posix_madvise(POSIX_MADV_DONTNEED) is a documented no-op on glibc), so I trust the measurement over a tidy causal story. It didn't stop the 211K OOM either way, because the 78.71 GiB artifacts copy is resident regardless.

(aside) GB10 unified memory makes RSS lie. During the degraded state, the process RSS was only 791 MiB — which looks like a process using no memory at all. It isn't. The weights live in unified memory, which doesn't count toward RSS. If you judge memory pressure by RSS on this box you'll conclude everything's fine while it's actively OOMing. Judge by free available plus the journal's batch-fit lines, not RSS.

Takeaways

Where the time went. Not the rollback — that was one config swap. It went into two detours that each looked like the fix: dropping context to dodge managed KV, and removing --warm-weights. Both moved the wall without removing it, and both would have shipped as "solved" if I'd stopped measuring. The cause only surfaced when one offhand fact — "the old engine never OOM'd on the same flags" — collapsed the whole flags-and-context theory in a sentence.

Reusable diagnostics. Separate the symptom you feel from the number that explains it. I felt "slow"; the number was 1.4 tok/s at 5% GPU, which reads as waiting, not computing — memory, not compute. And change one variable at a time: same flags, same context, same model, only the engine differs — so the engine is the cause. When your box has unified memory, throw out RSS and read free — RSS said 791 MiB while the thing was drowning.

The general principle. A headline tok/s number and a stable service are not the same product. Entrpi's +4 tok/s was real and was bought with the exact memory headroom a big request needs to not crash. For an always-on agent brain, the metric that matters isn't peak decode — it's whether it ever goes dark. A faster engine that OOMs a few times a day is slower, in the only way a user feels, than a slower engine that never does. And performance gains usually hold only within an envelope: Entrpi is sized for ~16K context; I pushed it 20-30× past that and then acted surprised. Check the workload envelope an optimization targets before you build on top of it.

---

Also in this series — DeepSeek-V4-Flash on DGX Spark

• Part 9 — Swapping the ds4 Engine for a Free Half-Generation Speedup — the prequel: how I got to 20 tok/s in the first place.
• Part 8 — Evaluating the Optimization Claims