Case type: Improve a metric

1IMPROVE-METRIC SPINE23  1. DEFINE        exact formula, window, population, exclusions4  2. NORTH STAR    one outcome metric tied to value creation5  3. DRIVER TREE   north star → drivers → inputs you can move6  4. DIAGNOSE      which node is soft vs already healthy (data asks)7  5. BETS          2–4 interventions mapped to nodes (not random ideas)8  6. PRIORITIZE    impact · confidence · effort · risk; sequence9  7. INSTRUMENT    what you must log before/during experiments10  8. GUARDRAILS    metrics you refuse to burn (trust, margin, latency)11  9. PLAN          30/60/90 or Q1/Q2 with kill criteria per bet1213  Anti-pattern: 12 feature ideas with no tree and no sequencing.

1MINI DRIVER TREE (say this shape out loud in minute 3)23  North star4    → driver A / driver B / driver C5         → inputs you can ship against6  + guardrails you refuse to burn78  Then: "I'll ask for data on A vs B vs C to find the soft node9  before I propose bets." That sentence alone is mid→senior.

1DRIVER TREE (sketch aloud)23  WAL4  ├── New WAL  (activation of new signups into a weekly habit)5  │     ├── Signup → first session6  │     ├── First session → week-1 return7  │     └── Acquisition mix quality (channel × intent)8  ├── Retained WAL (users active last week who return)9  │     ├── Habit loops (session frequency, streak, reminders)10  │     ├── Content fit (level match, exam relevance)11  │     └── Product reliability (crashes, paywall friction)12  └── Resurrected WAL (churned → return)13        ├── Winback campaigns14        └── Lifecycle triggers (exam date proximity)1516  Guardrails (not in WAL): paid conversion; learning outcomes;17  complaint/refund rate; D1/D7 notification opt-out.

1DIAGNOSIS → BETS MAPPED TO NODES23  Soft nodes: week-1 activation (esp. paid social); ongoing retention habit;4  content supply is NOT the constraint (library up, usage not).56  Bet 1 — Activation path rewrite (maps to New WAL / first→week-1)7    · Goal-based onboarding: exam date + weak topics → 7-day plan8    · First-session success event: complete one diagnostic + one lesson9    · Holdout: old onboarding1011  Bet 2 — Habit loop for retained (maps to Retained WAL)12    · Exam-date-aware weekly plan + progressive difficulty13    · Smart reminders only when plan is off-track (guardrail: opt-out)1415  Bet 3 — Acquisition quality (maps to New WAL mix)16    · Pause worst paid social creatives by D7 WAL CAC17    · Shift budget to high-intent SEO/exam pages + referral of passers1819  Bet 4 — DEFER: more content volume; AI tutor chatbot as top bet20    · Content already oversupplied; chatbot is a later retention enhancer21      once path-to-plan is stable (unlock: activation experiment wins)

1METRICS, EXPERIMENTS, GUARDRAILS23  North star: weekly active learners (defined)4  Primary drivers to move in Q1:5    · Signup → D7 WAL rate (overall and paid-social slice)6    · W1→W4 retention for new cohorts7  Input metrics: diagnostic completion; plan adoption; reminder CTR8  Guardrails: paid conversion rate; notif opt-out; refund rate;9              learning outcome proxy (practice accuracy trend)1011  Experiment that WOULD move WAL: onboarding that raises D7 WAL 15%12  relative on paid social without hurting refunds.13  Experiment that would NOT (false friend): homepage copy A/B that lifts14  signup but not D7 WAL — tree says activation after signup is soft, not15  raw signup volume alone.1617  Kill criteria: if Bet 1 shows <3% relative lift on D7 WAL after 4 weeks18  powered test → stop polish, revisit diagnostic of soft node.

1WEAK2  'Engagement is flat so I'd add streaks, push, a referral program, stories,3   AI tutor, more courses, gamification, and influencers. Success = higher4   DAU.' No tree, no diagnosis, no guardrails, no sequencing.56STRONG7  Define WAL → tree → data asks → soft nodes (activation + retention, not8  content supply) → 3 bets mapped to nodes → Q1/Q2 sequence → experiments9  with kill criteria → guardrails on revenue and trust.

1SPOKEN SENIOR ANSWER — Learning app WAL flat YoY (~12 min)23  [0:00–1:30 DEFINE + THESIS]4  "I'll treat this as improve-a-metric. Weekly active learners means unique users5  with at least one learning session of five-plus minutes in a week. Freemium6  with paid exam packs. Flat WAL with more content and more marketing spend7  usually means activation or retention leaks faster than top-of-funnel refills.8  Content volume is rarely the constraint when the library already grew. I will9  not open with streaks and push notifications."1011  [1:30–4:00 DRIVER TREE + DATA ASKS]12  "WAL splits into new WAL, retained WAL, and resurrected WAL. New WAL hangs on13  signup to first session, first session to week-one return, and acquisition mix14  quality. Retained hangs on habit loops, content fit, reliability. Resurrected15  hangs on winback and exam-date triggers. Guardrails outside WAL: paid16  conversion, learning outcomes, refunds, notification opt-out. Data asks in17  order: split WAL three ways; cohort retention YoY; activation signup to D7 by18  channel; session quality; seasonality and competitors. Suppose you return:19  retained down, new slightly up from paid UA, D7 soft on paid social, library20  up forty percent unused — soft nodes are activation and habit, not supply."2122  [4:00–7:30 BETS + SEQUENCE]23  "Bet one: activation path rewrite — exam date and weak topics into a seven-day24  plan; first success equals diagnostic plus one lesson; holdout old onboarding.25  Bet two: habit loop for retained — exam-aware weekly plan, smart reminders26  only when off-track with opt-out guardrail. Bet three: acquisition quality —27  pause worst paid social by D7 WAL CAC; shift to high-intent SEO and passer28  referral. Bet four deferred: more content volume and AI tutor as top bet —29  unlock when activation wins. Q1 is bets one and three plus instrumentation.30  Q2 is habit loops, then a narrow AI practice coach on one exam vertical."3132  [7:30–10:00 EXPERIMENTS + GUARDRAILS + XFN]33  "Primary drivers: signup to D7 WAL, W1 to W4 retention. Kill if bet one shows34  under three percent relative lift on D7 after a powered four-week test — stop35  polish, revisit diagnosis. False friend: homepage copy that lifts signup but36  not D7 WAL. Real friend: onboarding that lifts D7 on paid social without37  refunds. XFN: reframe UA KPI to D7 WAL CAC so install vanity dies. Content38  tags to exam plans, not volume vanity."3940  [10:00–12:00 CLOSE]41  "Plan: stop treating content and spend as the answer; fix activation and mix;42  then habit; AI later with unlock criteria. North star WAL with trust and43  revenue guardrails. If leadership forces AI in Q1, I still instrument the44  activation path and refuse to call the project done on demo quality alone."

1PRESSURE Q → SENIOR REPLY23  Q: "Give me ten growth ideas in two minutes."4  A: I can list three bets on the activation node — not ten orphans. Ideas5     without a tree node are theater.67  Q: "Leadership wants DAU, not WAL."8  A: I'll map DAU to learning-session quality or show how DAU gamifies opens.9     Prefer WAL as defined; if forced, add session-quality guardrail.1011  Q: "What if retention is fine and acquisition is the soft node?"12  A: Then I reverse sequence — mix and channel quality first, not onboarding13     rewrite. Tree first, ego second.1415  Q: "Ship more notifications to juice WAL."16  A: Only with opt-out and refund guardrails; measure D7 WAL not sends.1718  Q: "How long until we know?"19  A: Activation test: four weeks powered. Mix shift: two weeks of CAC by D7.20     Habit: longer cohort read — don't call habit won on week-one opens.2122  Q: "Name an experiment that would not move the north star."23  A: Signup-only copy test not powered on D7 WAL — classic false friend.2425  Q: "Content team says library growth is the strategy."26  A: Library already +40% with flat WAL. Tag content to exam plans and measure27     completion of plan items — volume without path is inventory vanity.

1SAY THE TREE OUT LOUD (2 minutes of airtime)23  'North star is weekly active learners — five-plus minute session.4   Under that: new, retained, resurrected.5   Under new: signup to first session, first to week-one, channel mix.6   Under retained: habit, content fit, reliability.7   Guardrails: paid conversion, outcomes, refunds, opt-out.8   Soft node from data: activation on paid social and retained habit —9   not content supply. Therefore bets hang only under those nodes.'1011  If you cannot speak that without notes, redraw the tree daily until you can.