← All questions
MediumAI MLTechnical
A model's live scores change with the composition of each request batch. The estimator weights are identical to training. What transformation bug would you test?
1Give yourself 5 minutes
2Answer out loud, not in your head
3Then compare with the answer below
0
Reference answer
Then expect these follow-ups
What if average scores look similar but individual vectors differ?
Free to read · better with Enzo
Practice this out loud with Enzo
Enzo runs it as a mock interview, pushes back with follow-ups, and grades you on the rubric.
Next question
- An AI system regressed after a prompt change. How would you debug it?
- How would you debug an AI deployment using a customer’s private data?
- What causes GPU out-of-memory errors during inference, and how do you respond?
- A booking worker crashes after a provider commits but before the local result is stored. The provider's idempotency record expires tomorrow. Write a recovery plan for today and for next week.
- Precision falls after a new region launches, while latency and feature missingness remain stable. Mature labels show the old region unchanged. What is the next diagnosis?
- A reproduction matches sample IDs and logits, but loss and gradient norms are exactly three times larger for a three-example batch. What do you inspect before another full run?