Lesson 4 of 4 · 60 min
Minimum useful still needs a trust boundary
Define a pilot slice with access, recovery, and operational ownership.
A minimum product is small in scope. It still needs to tell the truth about what it does with a user's data and actions. A pilot can omit advanced reporting, but it cannot quietly expose another customer's files. It can require manual recovery, but someone must know when recovery is needed and how to perform it.
Identify the irreversible or sensitive boundary in the first workflow. Examples include charging money, sending messages, publishing content, and storing private documents. Give those actions explicit authorization, safe operation identity, and an audit trail appropriate to the product. Reduce the number of such boundaries when designing an experiment. A local fixture can answer a learning question without accepting real customer data.
Reliability targets should follow user impact. A daily report may tolerate a delay of minutes, while a checkout confirmation cannot tolerate a misleading success state. Google SRE's guidance connects service objectives to user outcomes and decisions about reliability work. For a small team, begin with a small number of measurable outcomes and an owner who will act when they fail.
A feature flag can limit exposure, but it is not a security boundary. The server still needs permission checks. A flag can stop new work while existing work drains; it may not undo a completed effect. Record what turning it off actually does. A rollback plan is incomplete if the new version changed stored data in a way the old version cannot read.
Worked example
A fictional pilot allows three named customer accounts to upload weekly support exports. The minimum trustworthy slice validates file size and format, stores each file under account ownership, runs bounded jobs, and offers deletion through a controlled process. The operator sees failed jobs and oldest pending age. The pilot objective is that 95 percent of valid exports complete within five minutes, a provisional teaching target.
The stop control prevents new uploads and job starts. It does not delete stored results or pretend running work vanished. A recovery note identifies who can inspect failed jobs, which identifiers are safe to log, and how to confirm that a restored result belongs to the correct account.
Name the user promises
A trustworthy pilot should make a short list of promises that can be observed. The list is smaller than a full product specification, but each promise needs a mechanism and an owner. Separate access, correctness, recovery, and timing so a fast result does not conceal a privacy failure.
| Promise | Mechanism | Evidence before pilot |
|---|---|---|
| Only permitted accounts access files | Server-side ownership and action checks | Two-account retrieval test |
| One upload intent creates one logical job | Durable operation identity | Lost-response replay test |
| Incomplete output is not called complete | Prepared private content plus guarded publication | Interrupted-attempt schedule |
| Failures are recoverable | Owned queue/status and documented action | Known failed fixture recovered |
| Delay is visible | Completion objective and oldest-pending age | Timestamped valid-work sample |
This is an original pilot review matrix. Its mechanisms derive from the linked technical documentation; the choice of promises follows the fictional workflow. Do not imply that one employer's interview guide mandates this matrix.
A pilot with three customers can use a manual recovery step, but it needs a reliable trigger. If no one sees the failure, a recovery runbook is only an unused document. Identify how the operator learns about failed or old pending work, what action is safe, and which evidence confirms the repair.
Define the objective precisely
The provisional objective says 95 percent of valid exports complete within five minutes. Define the population before measuring. A valid export is one accepted under the input contract; malformed files rejected before acceptance do not belong in its completion denominator. Accepted work that later fails or stalls must not disappear from the count merely because it has no success event.
A small teaching window contains 100 accepted valid exports. Ninety-two finish within five minutes, four finish later, three fail, and one is still pending beyond five minutes. Timely completion is 92/100 = 92 percent, below the 95 percent objective. Counting 92/96 completed exports would hide failed and stalled outcomes. The exact measurement window and treatment of work near its boundary must be stated.
For a live pilot, avoid declaring stable reliability from a tiny denominator. One failure among twenty jobs changes the rate by five percentage points. The target guides a decision and an investigation; it is not a claim that the service has a proven long-run rate after one small sample.
Write the stop-control contract
code
1Control: pilot-export-admission=false2New uploads: rejected with a clear temporary-unavailable state3Queued jobs: held, with durable identity retained4Running jobs: finish under bounded execution, unless separately cancelled5Completed results: remain subject to access and retention policy6Operator: inspect held/failed jobs before enabling new startsThis is one valid drain policy. A different product may cancel running work, but it must implement cancellation and expose the actual outcome. Turning off a UI flag alone does not stop direct API calls or worker starts. Enforce the admission rule at the relevant server and worker boundaries.
A stop control also differs from rollback. If a release writes data the previous version cannot understand, redeploying old code can fail. Compatibility must be designed before the release, often by adding new fields first, migrating with verification, and removing old fields only after readers no longer require them. A feature flag cannot restore data that was destructively removed.
Plan for the only operator being absent
A one-person team should choose a failure policy that remains bounded without immediate attention. Hold new work when a critical dependency is unavailable or a budget is exhausted. Preserve status and identity so recovery is possible later. Do not promise instant support while depending on an operator who may be offline.
Manual recovery can begin with a small checklist: identify the authorized account/job, classify the failure, inspect the durable outcome, choose replay or new-input action under the contract, and verify the result through authorized retrieval. Keep sensitive payloads out of broad logs. The operator needs enough evidence to act, not an uncontrolled copy of every customer file.
Misconceptions and a second exercise
One misconception is that a limited rollout reduces access-control requirements. A small audience reduces exposure volume, not the correctness of account isolation. Another is that restoring old code restores old data. Code rollback and data recovery have separate boundaries.
Exercise: the operator is offline for twelve hours, the provider is unavailable, and twenty jobs are queued. Propose a bounded state. Hold jobs with durable identities, prevent unbounded paid retries, display accurate waiting status, and alert through the defined channel for later review. Do not create new jobs on each user refresh. Award one point for each of those four behaviors and one for explaining what resumes when the provider recovers.
A minimum trustworthy slice is not a full enterprise platform. It is a small outcome whose essential promises remain true when one ordinary failure occurs. State the manual parts openly and make their limits visible before inviting real user dependence.
Exercise and solution
The team wants to save a day by using one public storage URL for all pilot files. Explain the decision. The small user count does not remove the privacy boundary. Use account-authorized retrieval or avoid real private data during the experiment. Award one point for the boundary, one for a valid smaller alternative, and one for avoiding a promise that obscurity alone protects files.
Interview probe and wrap-up
Which reliability work can wait until after the pilot? A strong answer distinguishes expensive automation from essential recoverability. Manual recovery can be acceptable if it is bounded, documented, and owned. Follow up with the only operator being offline. A weak answer treats pilot as permission to ignore failures. Keep the product small while preserving the rules that make users able to trust it.
Sources
docsNext.js data security guidancenextjs.orgdocsGoogle SRE service-level objectivessre.googledocsAndrew Farries: expand/contract migrations, PGConf NYC 2024postgresql.usdocsPostHog feature-flag documentationposthog.comCheckpoint
100 valid accepted exports:92 timely,4 late,3 failed,1 overdue. Timely completion rate?
A95.83 percent from completed jobs only.B96 percent because eventual completion counts.C92 percent.D99 percent because only one remains pending.
Sign up free to answer and see whyCheckpoint
A pilot flag hides upload controls. What still enforces admission and privacy?
AThe UI flag alone.BAn opaque URL without access checks.CA note limiting the pilot to three users.DServer admission checks plus current account/resource authorization.
Sign up free to answer and see whyCheckpoint
The server now rejects new jobs. Two queued jobs are held. One running job is allowed to finish under the drain policy, and its result is not ready. Which UI state follows the implemented policy?
AShow all three jobs as paused, because new admission is disabled.BShow two held jobs and one running job; show a result only after its completion and access checks pass.CShow the running job as cancelled now, then replace that status with success if it finishes.DHide the running job until the stop control is removed, while showing the two held jobs.
Sign up free to answer and see whyCheckpoint
A release removed fields required by the old version. What does a feature flag alone fail to do?
ARestore deleted data and old-reader compatibility.BLimit new feature exposure.CChange some code paths.DStop a guarded admission route.
Sign up free to answer and see whyCheckpoint
The only operator is offline during a dependency outage. Best pilot behavior?
AMark every queued job succeeded for continuity.BRetry indefinitely because volume is small.CCreate a new job on each refresh.DHold durable work under bounded retry/budget rules and show accurate status.
Sign up free to answer and see whyCan you define a pilot's access, completion, recovery, and stop-control promises and measure them without dropping failures from the denominator? State the relevant identifiers, failure boundary, and evidence in your own words before selecting your confidence.
Not yetGetting thereConfident
Sources
Free to read · better with Enzo
Learn it with Enzo
Save your progress, answer the checkpoints, and let Enzo quiz you on what you just read.