DeepSeek V4 Flash Serves 304B Unquantized on One AMD MI300X

A published deployment stack runs the MIT-licensed DeepSeek-V4-Flash-0731 — 304B parameters, roughly 156GB unquantized — entirely inside one MI300X’s 192GB of HBM3. Author-reported figures: 168.6 tokens per second single-stream decode, 7.9 to 8.5K tokens per second tuned prefill, and 830 tokens per second on a 64-stream burst with no OOM. Getting there required fixing AMD’s FNUZ FP8 format, which diverges from the OCP standard by up to 2× in scale and fails silently, plus MoE routing corrections and GEMM tuning worth a 42 to 62 percent decode speedup. The stack pins a vLLM ROCm nightly and needs about 235GB of system RAM for its CPU KV tier.