09In a block-scaled FP8 GEMM like DeepSeek V3's, where are the scales applied, and why must accumulation be promoted to FP32?▼expertNewNVIDIAAMDOpenAI○ sign inFine-grained scales let E4M3 carry every tensor, but they change along the reduction axis, so the tensor cores cannot apply them. The answer walks the main loop where they do get applied, prices that work against the MMA, and explains what Blackwell moves into hardware.Open full answer →
07How are DeepSeek V3's multi-token prediction modules trained, and how does the engine turn that one MTP module into a speculative drafter?▼hardNewNVIDIATogether AIFireworks AI○ sign inDeepSeek V3 ships a drafter it trained alongside the model for free. The answer walks the MTP module's wiring and loss, prices a draft step against a 37.6B-active target, and reproduces the reported 1.8x from first principles.Open full answer →