Lessons
1The experiment is larger than the model file35 min read
Write a manifest that identifies a result and its dependencies.
- →Record enough experiment state to reproduce a comparison
2A checkpoint is a continuation contract35 min read
Explain which state is required to resume an optimizer trajectory.
- →Resume training with optimizer and data state
- →Track data order and preprocessing versions
3A seed is one control, not a reproducibility guarantee35 min read
Design separate tests for repeatability and seed robustness.
- →Control randomness without claiming universal determinism
- →Separate reproducibility from robustness across seeds
4Prove the optimized implementation matches a reference35 min read
Design a small reference test for a vectorized operation.
- →Verify an implementation against a small reference
Skills in this course
- 01Record enough experiment state to reproduce a comparisonRecord enough experiment state to reproduce a comparison.
- 02Resume training with optimizer and data stateResume training with optimizer and data state.
- 03Control randomness without claiming universal determinismControl randomness without claiming universal determinism.
- 04Verify an implementation against a small referenceVerify an implementation against a small reference.
- 05Track data order and preprocessing versionsTrack data order and preprocessing versions.
- 06Separate reproducibility from robustness across seedsSeparate reproducibility from robustness across seeds.