Flash Attention: The IO-Aware Fix

Standard attention is fast on paper and slow in practice. Learn why modern GPUs are bandwidth-bound, how the online softmax trick enables exact tiled attention, and why recomputing beats storing.