Bias-variance trade-off

Expected test error decomposes into irreducible noise plus bias-squared plus variance. High bias comes from under-fitting (too-strong prior, too few features, oversmoothed model); high variance comes from overfitting (deep nets without regularization, decision trees grown deep, k-NN with small k in high dimension). The trade-off shifts with hypothesis-class capacity, training-set size, and noise floor; resampling, regularization, ensembling, and early stopping each address one component. Cite concrete examples: "on a 50-sample fraud task a 6-depth GBM was high-variance - we switched to L2-regularized logistic with hand-crafted features and out-of-fold AUC rose 0.04".