FlashAttention 2 + 3 — Memory-Aware Attention Kernels

As of 2026-06-08

Senior field guide for FlashAttention 2 + 3: IO-aware attention kernel mechanics, v1/v2/v3 architecture evolution, Hopper-specific optimisations (TMA, warp-specialisation, FP8), BTP AI Core cost implications, and the diagnostic protocol when inference costs are higher than projected.