Lesson 4 of 8 · 55 min

Case type: Root-cause a metric change

Full diagnosis case: India checkout conversion −15% WoW — funnel decomp, hypothesis tree, falsification, reversible fix; plus churn/NPS 90-day drill.

Prescription before diagnosis is malpractice

"Conversion dropped 15% WoW in segment X" is an analytical / execution case. Senior answers build a funnel decomposition, a ranked hypothesis tree, and a falsification order — kill two hypotheses with data before proposing a fix. The anti-pattern is "add a notification" before you know which step broke.

Spot-it cue: dropped, declined, spiked, anomaly, "what happened," WoW/MoM change, segment-specific break. First move: validate the metric (definition, logging, bot filters), decompose the funnel, separate cohort vs period effects, rank hypotheses (product / system / external), falsify in cheap-to-expensive order.

How to solve root-cause cases

code

1ROOT-CAUSE SPINE23  1. VALIDATE METRIC   definition drift? logging bug? timezone? bot traffic?4  2. SCOPE             who/what/when/where — segment, platform, geo, channel5  3. FUNNEL DECOMP     which step rate broke (not just the end metric)6  4. COHORT VS PERIOD  new users worse, or everyone, or mix shift?7  5. HYPOTHESIS TREE   product · eng/system · external · measurement8  6. FALSIFY ORDER     cheapest / most likely first; kill before fix9  7. CONFIRM           triangulate with 2nd data source if possible10  8. FIX (OPTIONAL)    only after mechanism; smallest reversible change11  9. GUARDRAILS        north star vs the metric you patched1213  Anti-pattern: solution monologue in minute 3.

Interviewers at Meta-style analytical rounds are grading whether you can structure uncertainty. You will not have a warehouse in the room — ask for the cuts you need, state what each result would mean, and keep a living ranked list of hypotheses.

code

1SPEAK YOUR FALSIFICATION CONTRACT23  'If step X is flat and step Y is down, I'll prioritize hyps on Y.4   If provider dashboard != product DB, I stop product hyps and fix metering.5   If only new users in one channel broke, I chase mix/quality, not UI polish.'67  Each ask should come with: what I will conclude if high vs low.

Root-cause differs from improve-metric (L3) in the default verb: diagnose vs grow. You may still end with a plan, but only after a mechanism story. If the interviewer says "assume diagnosis is done, what do you ship?" — then switch gears explicitly.

Primary case prompt

Prompt: Checkout conversion on a mid-market B2B SaaS self-serve plan dropped 15% week-over-week for new signups from India (other geos flat). You own growth. Diagnose. Propose a fix only if the diagnosis supports it.

Worked strong answer

Validate. Confirm conversion = signup_completed → paid_or_trial_activated within 7 days (or whatever the team's definition is). Check deploy log, payment provider status, feature flags, analytics pipeline lag. Ask: did the denominator (signups) change composition? Assume interviewer says: definition stable, no global outage, India new-signup volume stable, conversion down 15% WoW only in that slice.

code

1FUNNEL DECOMP (India, new signups, last 4 weeks)23  Signup → Activate workspace → Connect payment → Start trial/paid45  Ask for step conversion rates WoW:6    Suppose: Signup→Activate flat7             Activate→Payment page  -3%8             Payment page→Success   -18%  ← break concentrates here9             Success→Day-1 use      flat among those who paid1011  Conclusion: mechanism is in payment completion, not top-of-funnel value prop.

Cohort vs period. Compare users who signed up this week vs last week (same age-on-product). If same-age cohorts convert worse, it is not just "we acquired more tire-kickers who never convert later." Also check device mix (mobile web up?), browser, UPI vs card, and partner referral tags.

code

1HYPOTHESIS TREE (ranked) + FALSIFY ORDER23  H1 MEASUREMENT  Payment success webhook delayed; metric undercount4      Falsify: reconcile provider dashboard vs product DB for same window56  H2 SYSTEM       Payment provider / UPI rails degraded in India7      Falsify: provider status + error codes by method; other products?89  H3 PRODUCT      Recent checkout UI change (address fields, tax, plan picker)10      Falsify: ship log + flag exposure; compare treatment vs holdout if any1112  H4 MIX SHIFT    New channel with low intent labeled as same segment13      Falsify: conversion by UTM/source within India; device mix1415  H5 EXTERNAL     Bank/regulatory or festival week cash constraint16      Falsify: calendar + peer benchmarks; prior-year seasonality1718  H6 PRICING FX   INR price display / FX rounding confuses users19      Falsify: session replay samples; support tickets with "price" tags2021  Order: H1 → H2 → H3 → H4 → H6 → H5 (cheap/internal first, fuzzy external last)2223  SUPPOSE: H1 killed (DB matches provider). H2 partially live: UPI failure24  rate +12pp. H3: checkout redesign shipped Tue to 50% — failures concentrated25  in treatment where UPI secondary button buried.

Synthesis. Primary mechanism: interaction of a checkout redesign (UPI de-emphasized) with India payment-method mix. Secondary: elevated UPI failures. Not a value-prop crisis; not "add more lifecycle email."

code

1FIX PLAN (only after mechanism)23  Immediate (hours):4  · Rollback or fix flag: restore UPI prominence for IN locale5  · Page payment provider on elevated UPI errors; enable backup rail if any67  Short (days):8  · Local method ordering by geo (UPI first in IN)9  · Error copy + retry for failed UPI with alternate method10  · Alerting: payment success rate by geo×method with 24h anomaly detect1112  Metrics:13    Recovery metric: payment success rate IN new signups → prior baseline14    North star: activated paid/trial workspaces (don't optimize only clicks)15    Guardrail: fraud/chargeback rate when retry paths expand1617  What you do NOT do: blast "complete your purchase" push to all India users18  before rails work — multiplies support load without fixing mechanism.

Communication pattern under pushback. If the interviewer says "just tell me what you'd ship," answer: "Smallest reversible fix for the confirmed mechanism is restore UPI prominence and page the provider — growth email comes after rails recover." You still lead with mechanism, but you give them a ship list.

Companion drill: NPS 22 / 60% annual churn (90 days)

Prompt: You inherit a B2B workflow tool with NPS 22 and ~60% annual logo churn. First 90 days? This is root-cause + triage, not a feature roadmap fantasy.

code

190-DAY RETENTION ROOT-CAUSE SKETCH23  Days 1–14  INSTRUMENT + SEGMENT4    · Churn reasons from exit interviews / CRM lost reasons5    · Cohorts by segment (SMB vs mid-market), champion role, use case6    · Activation events correlated with 6-month retention78  Days 15–45 DIAGNOSE TOP CHURN DRIVERS (pick 1–2)9    Typical: failed activation; single-threaded champion leaves;10    value not visible to economic buyer; reliability incidents;11    pricing cliff at renewal1213  Days 45–90 FIX THE DOMINANT MECHANISM14    · If activation: time-to-value playbooks (see L2 B2B beat)15    · If multi-thread: expand seats of engaged roles before renewal16    · If reliability: error budget + exec-visible uptime17    · Explicitly DEFER: rebrand, net-new product line, "AI everywhere"1819  Success: churn reason mix shifts; logo retention +NPS on the fixed segment20  Anti-pattern: 40-item roadmap that ignores why logos leave.

Weak vs strong

code

1WEAK2  'Conversion dropped so I'd improve the landing page, add discounts, send3   more emails, and A/B test the button color. Also add AI recommendations.'45STRONG6  Validate metric → funnel step isolation → ranked hypotheses → kill two with7  data → mechanism story → smallest reversible fix → monitoring so it can't8  silent-fail again.

Interview ways (root-cause)

01"X dropped 15% — what happened?" → Validate, scope, funnel decomp, cohort vs period, hypothesis tree, falsify order, then fix.
02"What data do you need?" → Ask for cuts that kill hypotheses (step rates, geo×method, flag exposure), not a data lake tour.
03"What's your top hypothesis?" → Give a ranked list with what would kill #1 — not a single stubborn theory.
04"Should we ship a growth feature?" → Not until the broken step is identified; growth features on a payment bug waste the quarter.
05"How do you avoid measurement theater?" → Reconcile second source (provider, server logs) when the metric is money-adjacent.
06"Guardrail vs north star?" → You may patch payment success (driver) while watching activated paid workspaces and fraud.

Full spoken senior answer (~12 minutes)

code

1SPOKEN SENIOR ANSWER — India checkout conversion −15% WoW (~12 min)23  [0:00–1:30 VALIDATE + SCOPE]4  "Root-cause case, not a growth brainstorm. First I validate the metric:5  conversion equals signup completed to paid or trial activated within seven6  days — or your team's definition. Check deploy log, payment provider status,7  feature flags, analytics lag, bot filters. Scope: India new signups only;8  other geos flat; volume stable. If definition or logging moved, I stop product9  stories and fix measurement. Assume you say definition stable, no global10  outage, India volume stable, conversion down fifteen percent WoW."1112  [1:30–4:00 FUNNEL + COHORT]13  "Funnel: signup, activate workspace, connect payment, start trial or paid.14  I ask step rates WoW. Suppose activate is flat, payment page to success is15  down eighteen percent — break concentrates in payment completion. Cohort vs16  period: same-age cohorts worse means not only tire-kickers converting later.17  Also device mix, UPI vs card, partner tags."1819  [4:00–8:00 HYPOTHESES + FALSIFY]20  "Ranked hypotheses. H1 measurement — webhook undercount; falsify with provider21  dashboard vs product DB. H2 system — UPI rails degraded; provider status and22  error codes. H3 product — checkout UI change buried UPI; ship log and flag23  exposure. H4 mix shift — new low-intent channel. H5 external — festival or24  bank events. H6 FX price display confusion. Order: H1, H2, H3, H4, H6, H5 —25  cheap internal first. Suppose H1 killed, H2 partial UPI failures up twelve26  points, H3 redesign shipped Tuesday to fifty percent and failures concentrate27  in treatment where UPI is secondary. Mechanism: redesign times India method28  mix, plus elevated UPI failures — not a value-prop crisis."2930  [8:00–11:00 FIX + METRICS]31  "Immediate: restore UPI prominence for IN locale; page provider on errors;32  backup rail if available. Short: geo method ordering, better error retry,33  alerting on geo times method success. Recovery metric: payment success to34  baseline. North star remains activated paid workspaces. Guardrail: fraud when35  retries expand. I do not blast complete-your-purchase email before rails work."3637  [11:00–12:00 CLOSE]38  "Diagnosis before prescription; kill two hyps with data; smallest reversible39  fix; monitoring so silent failure cannot recur. If you want only a ship list:40  restore UPI, page provider, alert — growth email after rails recover."

Follow-up pressure (root-cause)

code

1PRESSURE Q → SENIOR REPLY23  Q: "Just tell me what you'd ship in the first hour."4  A: After mechanism: restore UPI prominence and page the provider. Before5     mechanism: I will not invent a feature list.67  Q: "What if all funnel steps are flat but conversion is down?"8  A: Definition, window, attribution, or delayed success events — measurement9     first. Also check if denominator composition changed.1011  Q: "Could it be a competitor price cut?"12  A: Possible external H5 — after internal H1–H3. Check geo peer benchmarks13     and support ticket themes.1415  Q: "Why not survey users immediately?"16  A: WoW cliffs usually leave a quantitative footprint first; surveys later for17     residual qualitative.1819  Q: "Our CEO thinks it's the brand campaign."20  A: Brand rarely breaks one geo's payment step in a week. I'll still check mix,21     but payment localization is the prior.2223  Q: "How do you avoid analysis paralysis?"24  A: Timebox: kill two hyps today; ship reversible fix on best mechanism; keep25     a parked list for residual variance.2627  Q: "Should we pause all India acquisition?"28  A: Only if payment success is catastrophically broken and unfixable today.29     Prefer fix rails; pausing UA is a last resort that hides the wound.

Falsification order — speak the contract

code

1LIVING HYPOTHESIS BOARD (what seniors narrate)23  Ranked list (update live as data returns):4    1. Measurement mismatch     status: KILLED (DB == provider)5    2. UPI rail errors          status: LIVE (+12pp failures)6    3. Checkout redesign flag   status: LIVE (failures in treatment)7    4. Mix shift                status: OPEN8    5. External / festival      status: PARKED910  Each ask: 'If high, I conclude X; if low, I kill this hyp and promote Y.'11  That contract is the analytical senior signal Meta-style rounds grade.

When the interviewer role-plays incomplete data, do not invent tables. Ask for the cut that kills the most likely hyp, state the branch, and keep moving. Silence while you wait for perfect data reads junior.

articleExponent — Root Cause Analysis playlist (practice bank)Exponent

Companion habit: keep a personal 'hyp board' template in notes (measurement · system · product · mix · external). In the room, rewrite it for the metric at hand. Templates free working memory for judgment.

Google PM Execution: YouTube Watch Time DropExponent

Pause the mock at minute three. Write your falsification order before watching the rest. Compare: did you kill measurement/system before fuzzy external stories?

articleMeta analytical thinking interview (product execution)IGotAnOffer articleMeta Product Manager interview process overviewStellarPeers articleIGotAnOffer — product manager interview questions taxonomyIGotAnOffer articleStripe PM interview guide (analytical depth)Exponent

Checkpoint

Conversion dropped 15% WoW in India new signups. Your first three moves?

ALaunch a discount campaign and new onboarding UI immediately to recover revenueBValidate metric/logging, decompose funnel step rates for that segment, and start a ranked hypothesis tree with a falsification orderCSurvey 100 users about brand perception before looking at any funnel data

Checkpoint

Funnel data shows the break is payment page → success (−18%), earlier steps flat. Which hypothesis should rise?

AHomepage hero copy is unconvincingBPayment method UX, provider errors, or geo-specific checkout changesCNeed more top-of-funnel traffic from India

Checkpoint

Why falsify measurement/system hypotheses before fuzzy external ones?

AExternal factors never matterBInternal measurement and system checks are cheaper, faster, and often explain "mysterious" cliffs without storytellingCInterviewers forbid mentioning seasonality

Checkpoint

You killed two hypotheses with data. Interviewer asks for a fix. Best posture?

ARefuse to propose any fix until you have a 95% scientific certainty paperBPropose the smallest reversible change that addresses the confirmed mechanism, plus monitoring so recurrence is visibleCPropose five unrelated growth ideas in case the diagnosis is wrong

Checkpoint

In the NPS 22 / 60% churn companion drill, which first-90-days plan is strongest?

AShip a full product redesign and AI suite to "change the narrative"BSegment churn reasons and cohorts, identify 1–2 dominant mechanisms, fix those with measurable retention/NPS movement; defer net-new product linesCCut price 50% for all logos to buy retention

How ready are you to run a root-cause metric case with falsification order?

New to itGetting thereConfident

Root-cause locked

Validate → funnel decomp → cohort/period → ranked hyps → falsify → then fix.
Kill two hypotheses with data before prescribing.
Payment-localized breaks need payment-localized fixes, not lifecycle spam.
Companion: churn/NPS inheritance is root-cause + 90-day triage, not a feature dump.

Next: Critique a product — favorite-product case with taste and trade-offs.

Sources

Free to read · better with Enzo

Learn it with Enzo

Save your progress, answer the checkpoints, and let Enzo quiz you on what you just read.

Continue with Enzo Sign up free