Data

Prove MSE is non-convex for logistic regression

Asked at

Microsoft

1Give yourself 5 minutes

2Answer out loud, not in your head

3Then compare with the answer below

Reference answer

The logistic model is sigma(theta x). The MSE surrogate is J(theta) = 1/2n sum (sigma(theta x_i) - y_i)^2. J is non-convex because sigma is sigmoidal: composing a non-linear function with a squared error creates a landscape with multiple local minima at extreme theta (saturated predictions). Hessian is uninformative globally. This is exactly why cross-entropy is preferred for classification - the log sits inside the sum producing a convex objective. Equivalent sigma-in-MSE views go through X^T diag(sigma(1-sigma)) but with a (1 - 2y_i) factor that breaks sign-definiteness.

Then expect these follow-ups

What other surrogate losses are non-convex?
When might you still use MSE?

Free to read · better with Enzo

Practice this out loud with Enzo

Enzo runs it as a mock interview, pushes back with follow-ups, and grades you on the rubric.

Continue with Enzo Sign up free

Next question