Offline ranking improves, live clicks rise, and completed user tasks fall. How would you decide whether the candidate is useful?

Verify assignment, exposure logging, metric definitions, and uncertainty. Tie the decision to the intended task-completion objective. Investigate latency, repeated searches, and low-intent clicks as hypotheses with discriminating measurements. Keep predictive offline evidence separate from the effect of the live policy.