Design Recommendation Algorithm

Problem Statement

Design a recommendation system that surfaces personalised content or products to users based on their behaviour, preferences, and context. Think of it as the engine behind Netflix's rows, Amazon's "Customers also bought," or YouTube's "Up next."

The system must work for both media (movies, music, videos) and e-commerce (products) — the core architecture is identical. Where they diverge (signal types, catalogue size, engagement metrics), we'll call it out.

The reason this is hard isn't any single piece — it's that you need to solve three conflicting problems simultaneously: relevance (show good stuff), freshness (adapt quickly to new behaviour), and latency (do it in under 200 ms for every page load).

Requirements

Functional Requirements

Non-Functional Requirements

Requirements Tension

Scale Estimation

Every number is derived from an explicit assumption. The numbers drive the architecture — not the other way around.

API Design

GET /v1/recommendations?user_id=u_abc&context=homepage&page_size=50&exclude=item_1,item_2

Response 200:
{
  "request_id": "req_7f3a...",
  "items": [{
    "item_id": "item_882", "rank": 1, "score": 0.94,
    "explanation": { "type": "behavioral", "reason": "Because you watched Inception", "source_item_id": "item_331" },
    "metadata": { "title": "Interstellar", "image_url": "..." },
    "rec_source": "collaborative_filtering"
  }, ...],
  "model_version": "v2.4.1",
  "served_from": "cache_with_rerank"
}

context drives routing — homepage triggers broad exploration, cart page triggers complementary retrieval. request_id closes the feedback loop: rec → impression → click → training data. rec_source enables A/B testing per algorithm.

GET /v1/items/item_882/similar?user_id=u_abc&limit=20&diversity=0.3

Response 200:
{
  "source_item": "item_882",
  "items": [{
    "item_id": "item_990", "similarity_score": 0.91,
    "similarity_type": "content",
    "shared_attributes": ["sci-fi", "director:Nolan"]
  }, ...]
}

POST /v1/events
{
  "user_id": "u_abc", "event_type": "click", "item_id": "item_882",
  "context": { "page": "homepage", "position": 3, "request_id": "req_7f3a...", "session_id": "sess_x9f2..." },
  "event_value": { "watch_percent": 0.85 }
}

Response 202 Accepted  // fire-and-forget → Kafka

202 Accepted, not 200 — events go to Kafka asynchronously. context.position enables position bias correction in training. context.request_id joins the impression log to the event log, closing the feedback loop.

High-Level Architecture

Three lanes operating at three time scales, connected by a feedback loop. Batch (hours) produces quality. Stream (seconds) produces freshness. Serving (milliseconds) produces speed.

Deep Dive — Candidate Generation Strategies

Candidate generation is where recommendation systems are won or lost. The ranking model can only score what the candidate generator retrieves. Four approaches, in increasing sophistication:

1. Item-Based Collaborative Filtering

Compute item-item similarity from co-occurrence in the interaction matrix. Simple, fast (Redis lookup), and naturally explainable: "Customers who bought X also bought Y." But suffers from popularity bias (popular items are similar to everything) and sparsity (99.99% of item pairs have zero co-occurrence at 50 M items).

2. Content-Based Filtering

Match item attributes (genre, brand, price) to user preference vector. Solves cold start for new items (metadata exists on day one). But over-specialises — no serendipity — and requires clean metadata.

3. Matrix Factorisation

Decompose the sparse user-item matrix into dense user and item embeddings in a shared latent space. Score = dot product. Discovers latent patterns that co-occurrence misses. The bridge to modern methods — but can't incorporate side features.

4. Two-Tower Neural Model (Production Standard)

Two separate neural networks produce user and item embeddings. User tower ingests demographics, session context, click history. Item tower ingests category, price, text embeddings. Score = dot product. The towers are decoupled — item embeddings are precomputed and indexed for ANN search. One user tower forward pass (~2 ms) + one ANN lookup (~5 ms) = candidate retrieval from 50 M items in 7 ms.

sequenceDiagram
    participant C as Client
    participant S as Serving Service
    participant Cache as Rec Cache
    participant ANN as ANN Index
    participant CF as Item-CF Tables
    participant FS as Feature Store
    participant R as Ranking Model

    C->>S: GET /recommendations
    S->>Cache: Check precomputed recs
    Cache-->>S: Cache HIT (70%) → return
    Note over S: Cache MISS or stale ↓
    S->>ANN: User embedding → top 500
    S->>CF: Recent items → top 200
    Note over S: + Content-based 100 + Trending 100
    S->>S: Merge & dedup → ~1000 candidates
    S->>FS: Fetch features for 1000 items
    FS-->>S: User + item + cross features
    S->>R: Score 1000 candidates
    R-->>S: Ranked scores
    S->>S: Re-rank (diversity, rules, explore)
    S-->>C: Top 50 with explanations
        

ANN Index Internals (FAISS IVF+PQ)

Brute-force search across 50 M items: ~130 ms on CPU. Too slow. IVF clusters items into ~4096 cells via K-means, then searches only the nearest nprobe=64 clusters (~780 K items). PQ compresses each 128-d vector from 512 bytes to 32 bytes. Combined: ~3-5 ms for top-1000 at ~95% recall, using ~2 GB RAM.

Ranking Model (Stage 2)

A Deep & Cross Network scores the 1000 candidates with rich cross-features: user × item interactions (price vs. user average, time since last category purchase, position bias correction). Multi-objective: P(click), P(purchase), P(return). Scoring 1000 items: ~20-30 ms on CPU.

Production Systems Combine All Four

Multiple generators run in parallel — two-tower ANN (500 candidates), item-based CF (200), content-based (100), trending (100), personal history (100) — merge, dedup, and send ~1000 candidates to the ranker. Each generator covers the others' blind spots.

Key Design Decisions & Tradeoffs

Two-Stage Funnel vs. Single Full-Catalogue Ranking

Lambda (Batch + Stream) vs. Batch-Only

Two-Tower vs. Feature Cross Model for Candidate Gen

Daily Batch Retraining vs. Online Learning

Precomputed Cache: Full vs. Hybrid

What Can Go Wrong

The most dangerous failures are silent — HTTP 200, no errors, but recommendations are garbage for weeks.

Interview Tips

Evolution

Each stage is triggered by a specific bottleneck. Add complexity only when the numbers demand it.