Why multi-head attention instead of one big head?

With one full-dimension head, a single softmax over the whole d_model mixes all feature subspaces into one attention distribution; in practice that distribution becomes diffuse and the model can only attend to one combined signal at a time. Multiple heads project into independent subspaces and compute separate attention patterns in parallel; outputs are concatenated and re-projected. Empirically this lets the model attend to different syntactic/semantic relations in the same layer, e.g. one head tracking syntactic dependencies, another coreference. There is a parameter-count invariance (h small heads vs 1 big head costs roughly the same FLOPs) but a representational one: more heads means more patterns per layer. The trade-off: too many heads wastes compute; too few collapses representational capacity. Modern efficient variants like GQA/MQA share KV across heads to save memory with minimal quality loss, a great senior-level gotcha to mention.