Debug an actual LLM inference step in a 20–30 line Colab skeleton (Anthropic)

Treat the skeleton as a black box you cannot search-engine. Narrate your read of each line — what tensor comes in, what tensor goes out — and trace one prompt end-to-end on paper before touching code.

Build a checklist of likely defect classes: shape mismatch between hidden states and lm_head; off-by-one in position IDs; softmax temperature missing or applied twice; top-k applied before vs after temperature; masking on the wrong axis; NaN from log(0) in cross-entropy; jinja templating eating whitespace in chat templates. Diagnose via print-shape checkpoints inserted at module boundaries.

Stretch-time signal: notice when an apparent bug is actually a correct numerical-noise tolerance and skip chasing it. Crucial etiquette: stay narrative ("I'm scanning for shape errors first, then for sampling bugs"), never paste an external snippet, never silently invoke an LLM.