Why divide by sqrt(d_k) in scaled dot-product attention?

Scaled dot-product attention computes Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) V. As d_k grows, the dot products of unit-variance Q and K vectors grow in magnitude, pushing the softmax into regions of tiny gradient (the "softmax saturation" problem). Dividing by sqrt(d_k) keeps the input to softmax roughly unit-variance regardless of head dimension, which preserves gradient flow and helps avoid extremely peaked attention distributions during early training. Without it you'd see training instability: entropies collapse, one token dominates attention, and learning stalls. In practice, modern variants sometimes replace this with QK-norm or other normalizations for very large d_k, but the sqrt(d_k) factor remains the canonical default because it is dimension-aware without adding parameters. Senior candidates also mention that with sufficiently normalized embeddings, scaling matters less, but it is still kept for general robustness across head widths.