HardCodingSystem design

Design an on-device AI assistant

Asked atAGApple, Google

1Give yourself 5 minutes

2Answer out loud, not in your head

3Then compare with the answer below

Reference answer

Requirements: <50ms first token; <2GB memory; works offline; secure against model extraction.

Architecture: small SLM (3–8B) quantized int4/int8; ONNX Runtime / Core ML / TFLite; tool-call policy constrained to device-only actions; local vector store of user content; sync with cloud for escalation.

Eval: per-task quality vs the cloud baseline; latency; battery cost; cold-start.

Depth signals: token streaming cadence design; quantization's effect on quality for dialog tasks; a tier-based escalation policy.

Follow-up probes: How do you decide what stays on device vs escalates? How do you update the model?

Hiring now

Get the Apple, Google interview, not just the prep

landed applies to roles like these for you, on each company’s own site, with answers written for the role.

Get 5x interviews10 free auto-applies. No card.

Or practice this question first

Next question