Skip to content
← Attention Kernels / 21
hardNewNVIDIATogether AIFireworks AI

Which attention kernel will vLLM and SGLang actually run on an H100 and a B200, and what silently changes the answer?

An FP8-cache A/B on one card says FP8 is slower, and the same flags on an air-gapped B200 run slower than staging. Neither result is about FP8 or the GPU. The answer walks each engine's selection rule from source and lists the settings that move a deployment onto a different kernel without an error.

Updated Nov 2026 · Worked answers on inference mechanisms, with calculations, implementation details, and follow-up questions.

An FP8-cache A/B on one card says FP8 is slower, and the same flags on an air-gapped B200 run slower than staging. Neither result is about FP8 or the GPU. The answer walks each engine's selection rule from source and lists the settings that move a deployment onto a different kernel without an error.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 213 remaining answers · $25

Every worked answer and derivation in all 12 topics, every concept page, the full practice-test bank, and every code drill’s edge-case tests and reference solution.

6 months · One payment · No auto-renewal

UP NEXT ON YOUR JOURNEY
Next in this trackResearch hands you document masking plus sink tokens to serve. FlexAttention, or fork a FlashAttention kernel?Popular right nowWhy is batch-1 decode on a 70B model bandwidth-bound, and what is its latency floor on two H100s?Popular right nowDecode attention intensity is fixed at the query-to-KV head ratio. Derive that, then tell me what MLA does to it.
DISCUSSION · 0

Nobody has commented on this answer. Add a correction or another way to work it.