[Just for Fun — Advanced] I Tried to Undervolt My 2080 Ti — Then Learned Decode Doesn't Care About Watts

The short version: I wanted to save watts, and the card told me watts were never the bottleneck

When I put this modded 2080 Ti into service as a 24/7 agent brain (#1), one of the first things on my list was to undervolt it. The logic felt obvious: it's a card that's on all the time, undervolting drops the power draw and the heat, and on a Turing card you usually give up almost nothing in return. Free quiet, free efficiency.

So I went looking for how much speed I'd be trading away. And the answer turned out to be: roughly none — but not for the reason I expected. The reason is that for the work this card actually does most of the day, watts were never the lever in the first place.

This is a short one, and the punchline is a single idea worth keeping: decode is bandwidth-bound. Once you internalize that, a whole cluster of GPU tuning instincts — undervolt and you'll lose speed, overclock the core to gain it, push the power limit for more tokens — quietly rearrange themselves.

⚠️ A disclaimer before anything else: I want to claim I undervolted to 220W and measured no slowdown. I can't. When I went to check, the card was running at its 250W default — the undervolt I thought I'd set never actually took effect. So I don't have a clean paired 220W-vs-280W decode comparison. What I do have is the picture of what the card is doing under load, and that picture is enough to reason about where the bottleneck sits. I'll flag every "this is reasoning, not a measurement" moment as it comes up.

---

The intuition I started with (and why it's wrong here)

The folk model most of us carry for GPUs goes like this: more power → higher clocks → more work per second. Undervolt and you cap the clocks, so you go slower. Overclock and push the power limit, and you go faster. For a lot of GPU workloads — gaming, rendering, training — that's roughly true, because those are compute-bound: the chip's arithmetic units are the bottleneck, so feeding them more power and clock genuinely buys throughput.

LLM decode is not that kind of workload.

When a model generates text, it produces one token at a time, and for every single token it has to read the entire set of active weights out of VRAM, multiply through them, and emit one token. Then it does it again for the next token. The arithmetic per token is small; the data movement per token is enormous. So the thing that gates you isn't how fast the cores can compute — it's how fast you can stream the weights out of memory. That's memory bandwidth. A workload limited this way is called bandwidth-bound, and no amount of extra core clock or extra watts speeds it up, because the cores are already sitting around waiting for memory.

What the card was actually doing during decode

Here's what flipped my intuition: I watched the live GPU state while the model was generating, and it doesn't look like a card that's straining for power. It looks like a card that's waiting on memory.

• The power cap was active — the card was being held at its limit — but the core (SM) clock was bouncing around in the 1665–1845MHz range, well below its ceiling. A sub-ceiling core clock isn't proof of bandwidth-bound on its own (a power-capped GPU can settle below its boost ceiling for other reasons) — but combined with the next two signals, it fits. [memory]
• The memory clock was pinned flat at ~6800MHz [memory] — effectively maxed out against the card's rated memory clock ceiling of 7000MHz [live]. The memory subsystem was doing all it could; the cores were not.
• The chip was sitting at 47–52°C. Cool. Nowhere near a thermal wall — which means I'd be undervolting purely for quiet/efficiency, not to stop the card from throttling. [memory]

Put those three together and the story writes itself: the memory is the bottleneck, the cores are not, and the chip isn't even warm. That is the textbook profile of a bandwidth-bound workload.

Let me be precise about provenance, because it matters here. What I verified live in this session ([live]): the 250W default, the min/max power range of 100–280W, the 7000MHz rated memory-clock ceiling, and the idle snapshot below. From [memory] (the earlier tuning session, not re-measured while writing this): the under-load core/memory clocks — including that ~6800MHz figure — and the 47–52°C. I didn't re-capture the under-load clocks for this post because I didn't want to drive heavy inference through the live brain just to re-screenshot a value.

One thing I could verify live

I can at least confirm the power configuration is exactly what I said it was. Querying the card directly:

$ nvidia-smi --query-gpu=name,power.limit,power.default_limit,power.min_limit,power.max_limit,clocks.max.memory --format=csv
NVIDIA GeForce RTX 2080 Ti, 250.00 W, 250.00 W, 100.00 W, 280.00 W, 7000 MHz

So: the power limit is 250W, which is also the default — confirming the undervolt I thought I'd applied isn't in effect. The adjustable range is 100–280W, and the rated memory clock max is 7000MHz (the ~6800 I saw under load is right up against it). [live] This is the snapshot at idle; I'm not claiming the under-load clocks from a fresh capture, only the power envelope and the limits.

Why decode doesn't move, but prefill might

If decode is bandwidth-bound, where could extra power help? Prefill — the phase where the model reads your entire prompt at once before it answers a single token.

Prefill is a different shape of computation. Instead of one-token-at-a-time, it processes a whole batch of tokens in parallel, which keeps the arithmetic units genuinely busy. That makes prefill compute-bound (or at least far more compute-leaning than decode). So in principle, more power and a higher core clock can shave a bit off prefill. [memory]

But here's the deflating part: prefill is also the phase where, on this hybrid brain, a cache miss makes you re-process the entire prompt from scratch — and at 45–50K tokens that's a 200-second wait that no realistic clock bump touches (#5 is the whole story). The lever that actually moves prefill is cache behavior, not watts. So even on the one phase where power could matter, it's a rounding error next to the cache problem.

Net: there is no phase of the workload where pushing the power limit is the right lever.

The second tell: decode speed ignores context length

There's a clean, independent confirmation of "bandwidth-bound" sitting in the data from an earlier post, and it doesn't depend on watching clocks at all.

If decode were compute-bound, you'd expect longer contexts to slow it down — more tokens in the KV cache, more attention math per step. But that's not what happens. Across the whole range I measured, decode held at ~32 tok/s whether the context was 4K or 110K. [memory]

That's exactly the fingerprint of a bandwidth-bound process: the per-token cost is dominated by streaming the weights — a fixed amount of data regardless of how long your conversation is — so throughput stays flat as context grows. (To be precise: reading the KV cache does grow with context, so it isn't literally constant work; it's just that on this card, in this range, the weight-streaming term dwarfs it, which is why the curve reads flat.) Prefill, the compute-leaning phase, does the opposite — its throughput drops as the prompt gets longer, from ~624 tok/s at 4K down to ~360 tok/s at 110K. [memory] Two phases, two completely different scaling behaviors — and the decode one is flat, which is the bandwidth signature.

So I have two independent tells pointing at the same conclusion — the live clock/temp profile, and the flat decode-vs-context curve — and they agree. Neither one is a paired power experiment, but together they make the bandwidth-bound read hard to argue with.

What this means for tuning a card like this

Once you accept that decode is bandwidth-bound, the practical advice falls out:

• Undervolting should be close to free. If the cores are already idling below their ceiling during decode and the chip is at 50°C, then as long as a lower limit still keeps the memory clock near its ceiling, dropping the power limit should cost essentially nothing on the part of the workload you run most — you'd get the quiet and the efficiency without paying for it in tokens. I still owe myself a real paired test to prove the cost is zero (see the disclaimer); every indicator points to negligible, but I haven't measured it.
• Don't chase tokens with watts. Cranking the power limit toward 280W won't make decode faster, because the memory clock is already pinned. You'd be paying for heat you don't need.
• If you want decode faster, the only real lever is bandwidth itself — a card with more GB/s. This 2080 Ti's modding doubled the VRAM capacity (11→22GB) but left the bandwidth unchanged at ~616 GB/s (#1). More room for context and bigger models — but the same decode ceiling. Capacity and bandwidth are different axes, and the mod only moved one of them.
• The number that actually decides how the agent feels isn't a clock at all — it's TTFT, the prefill cache behavior (#5). Watts are a sideshow next to it.

The bottom line

I went in wanting to undervolt to save power and bracing to lose a little speed. The card's own behavior told me there was almost nothing to lose: decode is bandwidth-bound, gated by how fast weights stream out of VRAM, and during generation the cores were already coasting below their ceiling while the memory clock sat pinned and the chip stayed cool. More watts wouldn't help; fewer watts wouldn't hurt much.

The limits of this post, stated plainly: my undervolt never actually applied (the card held at 250W), so I can't show you a clean 220W-vs-280W paired decode comparison — that's still on the to-do list. What I can show is the shape of the bottleneck from the clock and temperature readings and from the flat decode-vs-context curve, and both say the same thing. So go ahead and undervolt your always-on card for quiet and efficiency; just don't expect watts to be the dial that makes tokens come out faster. On a bandwidth-bound workload, that dial isn't connected to much.

---

Also in this series:

• (related) Why 30 tok/s feels slower than 14: TTFT is the number you actually feel
• (related) Quantizing the draft cache backfired: a counterintuitive look at Qwen MTP
• (origin) I scored a 22GB-modded 2080 Ti — just enough to keep a 27B agent running at home