MediumAI MLTechnical screen10 min to answer

What is data leakage, and how can it invalidate an evaluation?

1Give yourself 10 minutes

2Answer out loud, not in your head

3Then compare with the answer below

Stuck? Show a way to structure it+

01Clarify the objective, user, constraints, and acceptable failure modes.
02Structure the response around train-test contamination, temporal leakage, duplicate data, audit.
03Compare credible alternatives and state the decision criteria.
04Finish with validation, monitoring, and what would change the decision.

Reference answer

strong answer

I would first define the outcome and acceptable error boundary. I would explicitly cover train-test contamination, temporal leakage, duplicate data, audit. I would select an approach only after comparing quality, latency, cost, safety, and operational complexity, then validate it with representative cases and production monitoring.

train-test contamination
temporal leakage
duplicate data
audit

Must be adapted to the interviewer’s scenario
Would benefit from concrete numbers and personal examples

alternative answer

A second valid approach is to begin with the highest-risk assumption and test it cheaply. I would explicitly cover train-test contamination, temporal leakage, duplicate data, audit. I would select an approach only after comparing quality, latency, cost, safety, and operational complexity, then validate it with representative cases and production monitoring.

train-test contamination
temporal leakage
duplicate data
audit

Must be adapted to the interviewer’s scenario
Would benefit from concrete numbers and personal examples

Grade yourself

train-test contamination

weight 30%

StrongExplains train-test contamination concretely and ties it to a decision or measurable outcome.

WeakMentions train-test contamination without implications, trade-offs, or evidence.

temporal leakage

weight 25%

StrongExplains temporal leakage concretely and ties it to a decision or measurable outcome.

WeakMentions temporal leakage without implications, trade-offs, or evidence.

duplicate data

weight 25%

StrongExplains duplicate data concretely and ties it to a decision or measurable outcome.

WeakMentions duplicate data without implications, trade-offs, or evidence.

audit

weight 20%

StrongExplains audit concretely and ties it to a decision or measurable outcome.

WeakMentions audit without implications, trade-offs, or evidence.

Then expect these follow-ups

What would your evaluation dataset contain?
Tests: depth
What if offline and online results disagree?
Tests: adaptability
How would you set the launch threshold?
Tests: adaptability

Hiring now

Machine learning engineer roles are open right now

landed applies to roles like these for you, on each company’s own site, with answers written for the role.

Get 5x interviews10 free auto-applies. No card.

AI/ML Engineer
Kaizen Labs · New York, NY
Senior ML Engineer, Perception
Clera · San Francisco
Machine Learning Engineer
Endava · Córdoba, Cordoba, Argentina

Or practice this question first

Next question