#vllm — Blog — ai-muninn

~ / blog / tag / vllm

❯ grep -r "#vllm" ~/blog

43 matches

datereadtitle
2026-09-2817m
[DGX Spark] Qwen3.8-Flash-Next on two DGX Sparks: 51.9 to 72.0 tok/s, and a draft vocabulary built from my own text
#dgx-spark #qwen3.8-flash-next #vllm #mtp
2026-09-1219m
[vLLM] Qwen3.8-Flash-Next TP=2 on Two DGX Sparks: 51.9 tok/s
#dgx-spark #vllm #tensor-parallel #qwen3.8-flash-next
2026-09-0623m
[Benchmark] Qwen3.8-Flash-Next NVFP4 on a DGX Spark: 41.7 tok/s, RAM for Traffic, Disk for the Dictionary
#dgx-spark #gb10 #vllm #nvfp4
2026-07-078m
DGX Spark in 2026: What Still Works, What Broke, and What I'd Run Today
#dgx-spark #gb10 #gemma-4 #vllm
2026-06-1310m
[vLLM] DiffusionGemma 26B NVFP4 on a DGX Spark: 158 tok/s, and why diffusion tok/s lies
#dgx-spark #gb10 #diffusiongemma #diffusion-llm
2026-06-1116m
[Benchmark] Qwen3.5-122B on DGX Spark — 2× faster
#qwen3.5 #dgx-spark #gb10 #gdn
2026-06-046m
[Benchmark] Gemma 4 12B Omni on DGX Spark: Weight-Only NVFP4 Beats W4A4 (and Keeps Multimodal)
#dgx-spark #gb10 #gemma-4 #nvfp4
2026-06-0110m
[Benchmark] NVFP4 W4A4 beats FP8 on a DGX Spark MoE: 67 vs 52 tok/s once CUDA graphs fire
#nvfp4 #w4a4 #fp8 #dgx-spark
2026-05-309m
NVFP4 is 1.5× FP8 on a DGX Spark — but it's compression, not the FP4 cores
#nvfp4 #fp8 #dgx-spark #gb10
2026-05-2110m
Round 2 EAGLE-3 retrain didn't break the ceiling — a 60-hour null-result writeup
#gemma-4 #abliteration #eagle-3 #speculative-decoding
2026-05-1614m
EAGLE-3 fine-tune against an abliterated Gemma 4 body — Round 1 flattens the acceptance curve (plus a measurement lesson)
#gemma-4 #abliteration #eagle-3 #speculative-decoding
2026-05-146m
30 lines of docker for +34% on DGX Spark: huihui Gemma 4 FP8 + vanilla MTP n=1 deployment recipe
#gemma-4 #abliteration #mtp #speculative-decoding
2026-05-0914m
Want MTP speedup on abliterated Gemma 4? Vanilla draft can't track the modified body
#gemma-4 #abliteration #mtp #speculative-decoding
2026-05-0615m
Liftoff: Gemma 4 hits 670 tok/s aggregate on DGX Spark (108 tok/s single-stream)
#gemma-4 #mtp #speculative-decoding #vllm
2026-05-0113m
[vLLM] Watching English Videos with DGX Spark: Nemotron Omni Multimodal on GB10
#nemotron-omni #multimodal #vllm #dgx-spark
2026-05-0114m
[vLLM] Nemotron 3 Nano on DGX Spark: 74.75 tok/s NVFP4 — 11.5% Past the Public Baseline
#nemotron-3 #nvfp4 #vllm #dgx-spark
2026-04-268m
[Benchmark] Abliteration Costs 1.85pp on Traditional Chinese — and 7.7pp on Trust Law
#tmmlu+#abliteration #traditional-chinese #qwen-3.6
2026-04-257m
[Benchmark] TMMLU+ Paired Eval: Qwen 3.6 35B Sweeps Gemma 4 26B 51-of-51 on Traditional Chinese
#tmmlu+#traditional-chinese #qwen-3.6 #gemma-4
2026-04-2214m
[Hands-On] Making NVFP4 17% Faster on GB10 with a Triton FP8 Bypass
#nvfp4 #fp8 #triton #dgx-spark
2026-04-218m
[Benchmark] NVFP4 Is a Trap on GB10: FP8 Wins by 32% (vLLM + SGLang Tested)
#nvfp4 #fp8 #dgx-spark #gb10
2026-04-207m
[Benchmark] Same Scaffold, Three Models: 16% → 38% → 48% on SWE-bench Lite
#swe-bench #gemma-4 #qwen-3.6 #scaffold
2026-04-1712m
[Benchmark] SWE-bench Lite 38.67% with a 26B Local Model — 0.33% from Claude 3.5 Sonnet Scaffolds
#swe-bench #gemma-4 #mini-swe-agent #vllm
2026-04-158m
[AI Agent] Gemma 4 26B Cleared a SWE-bench Lite Instance — After 28 Tries Across Two Days
#swe-bench #mini-swe-agent #gemma-4 #vllm
2026-04-137m
[AI Agent] Gemma 4 Went from 40 Errors to a 9-Step Bug Fix — by Switching One Thing
#swe-bench #gemma-4 #qwen-3.5 #openhands
2026-04-136m
[Benchmark] Gemma 4 on DGX Spark — Which Model Should You Pick?
#gemma-4 #dgx-spark #gb10 #benchmark
2026-04-138m
[DGX Spark] From Unboxing to Running: Complete Deployment Guide
#dgx-spark #gb10 #gx10 #vllm
2026-04-079m
[Benchmark] From 19 to 50 tok/s: We Quantized Gemma 4 E4B to NVFP4 Before Anyone Else
#gemma-4 #e4b #nvfp4 #fp8
2026-04-078m
[LLM 101 #1] Ollama vs vLLM: Two Ways to Run AI on Your Own Computer
#ollama #vllm #llm #local-ai
2026-04-056m
[Benchmark] Gemma 4 31B Dense on DGX Spark: 7 tok/s and the Bandwidth Wall
#gemma-4 #nvfp4 #vllm #dgx-spark
2026-04-058m
[Benchmark] vLLM vs Ollama on the Same Model: Why 30% Faster on GB10
#vllm #ollama #benchmark #dgx-spark
2026-04-059m
Gemma 4 26B-A4B on DGX Spark: 52 tok/s with NVFP4, skip the 31B
#gemma-4 #nvfp4 #vllm #dgx-spark
2026-04-026m
[DGX Spark] Overheating, 100W Power Cap, 30W Safety Mode — Complete Diagnostic Guide
#gx10 #gb10 #dgx-spark #power-delivery
2026-03-308m
[Benchmark] TurboQuant on GX10: Is 3-bit KV Cache Compression Actually Lossless?
#turboquant #kv-cache #quantization #vllm
2026-03-247m
[AI Agent] NemoClaw Without the Cloud: Swapping Nemotron for a Local Ollama Model
#nemoclaw #openclaw #openshell #ollama
2026-03-216m
[AI Agent] openclaw + 131K Context: When max_tokens Goes Negative
#openclaw #context-window #vllm #gpt-oss
2026-03-216m
[vLLM] FP8 KV Cache on GB10: Why Outputs Collapse into Repetition Loops
#vllm #fp8 #kv-cache #gb10
2026-03-1912m
[vLLM] Running a 120B Model on DGX Spark at 60 tok/s — Zero API Cost, Six Bugs
#dgx-spark #sm121 #vllm #gpt-oss
2026-03-197m
[vLLM] Qwen3.5-122B Runs. But at 14 tok/s.
#dgx-spark #sm121 #qwen3.5-122b #vllm
2026-03-1711m
[vLLM] Why Your DGX Spark Only Says "!!!!!": Debugging NVFP4 on SM121
#dgx-spark #sm121 #vllm #nvfp4
2026-03-1310m
[vLLM] Nemotron-3-Super-120B on a Single GB10: Full Day Debug Log
#dgx-spark #gb10 #sm121 #nemotron
2026-03-078m
[vLLM] Ollama's KEEP_ALIVE Is Silently Eating Your vLLM Headroom
#vllm #ollama #gpu-memory #dgx-spark
2026-03-067m
[vLLM] Don't Add --enable-chunked-prefill to SSM Models
#vllm #ssm #qwen #dgx-spark
2026-03-0511m
[vLLM] Qwen3.5-35B at 47 tok/s on DGX Spark: Ollama to vLLM Migration Guide
#dgx-spark #gb10 #vllm #ollama

← back to all posts