Design Recommendation Algorithm

Open almost any large app and the first screen is a row of things picked for you: products, videos, songs, articles. Behind that row is a system that must choose roughly 30 items out of 40 million for one specific person, in less time than it takes the page to paint, and then learn from whatever that person does next.

No model can score 40 million items per request at that speed. So the real design is a funnel. Cheap methods pull a few thousand plausible candidates from several sources, a heavier model scores those carefully, and a final pass adjusts the list for variety, freshness and business rules. Around the funnel sit the pieces that keep it honest: an event stream, a feature store, training pipelines and an experimentation platform.

Take a shopping and media app with 180 million daily users, a catalogue of 40 million items, and about eight recommendation calls per user per day across all surfaces. Each call returns 30 items from a funnel that ranks around 400.

Requests flow left to right through four steps. The recs API loads the user's features and session state, then calls several candidate generators in parallel, each with its own 25 ms deadline. Their results are merged, deduplicated and filtered against what the user has already bought or dismissed. A light pre-ranker trims the union to about 400, the ranking service scores those with the full model, and the re-ranker turns scores into a final page.

Two loops keep the path current. A fast loop runs through Kafka and Flink, updating session counters and recent-item lists in the feature store within seconds. A slow loop runs through the data lake, retraining models and rebuilding the ANN index on a schedule, then shipping new artefacts behind an experiment flag.

Writing the budget down early matters because each team will otherwise spend the slack on its own component. A rule that every call carries its remaining deadline, and every service gives up when it runs out, keeps one slow dependency from eating the whole request.

Item-to-item collaborative filtering counts how often two items appear in the same session, normalised so that globally popular items do not pair with everything. A nightly job builds, for each item, a list of its top 200 neighbours. At request time you take the user's last ten interactions, look up their neighbour lists and sum the weights. It is cheap, explainable and very strong for "next item" intent, but useless for an item nobody has touched yet.

For a product listed an hour ago there are no interactions, only a title, images, category and price. An encoder turns those into a vector, and the item becomes retrievable by similarity to things the user liked. Content retrieval rarely wins on its own, but without it new items would never get the first impressions they need to enter the collaborative signals.

Classic matrix factorisation learns one vector per user and one per item so that their dot product predicts interaction. It works, but a user who signed up today has no learned vector, and it cannot use context such as time of day or device.

The two-tower model fixes both. One neural network, the user tower, takes recent history, demographics and context and outputs a 128-dimensional vector. The item tower does the same from item attributes and IDs. Training pushes the vectors of real interactions together and sampled non-interactions apart, usually with in-batch negatives and a correction for popularity so the model does not just learn "show bestsellers". Because the towers never see each other's inputs, item vectors can be computed offline for the whole catalogue, and only the user tower runs per request.

Exact nearest-neighbour search over 40 million vectors is about 5 billion multiply-adds per query. Approximate indexes trade a little recall for orders of magnitude in speed. Two families dominate:

• IVF with product quantization. K-means splits the space into, say, 16,384 clusters. A query visits the 64 closest clusters only, and each vector inside is stored as a 32-byte compressed code whose distances are computed with lookup tables. The whole index fits in about 2 GB of memory, so every serving node holds a full copy.
• HNSW graphs. A layered proximity graph walked greedily from coarse to fine. Recall is higher at the same latency, but memory use is several times larger and incremental deletes are awkward.

Tune for recall at 800 above about 95% against exact search, measured on a held-out set. The ranker can only reorder what retrieval gives it, so lost recall here is invisible downstream and very expensive.

The ranking model sees each user and candidate pair together, so it can use cross features retrieval cannot: how often this user bought from this brand, the price relative to their usual spend, whether the item is in their cart. A typical shape is a deep network with feature-crossing layers and several output heads, predicting click, add to cart, purchase and "not interested" at once.

The final score is a weighted blend, for example 0.1 x p(click) + 0.4 x p(cart) + 1.0 x p(buy) - 2.0 x p(dismiss). Those weights are product decisions tuned by A/B tests, not by the model. Keeping the heads separate means you can change the blend without retraining.

A list sorted purely by score tends to show eight near-identical phone cases in a row. The re-ranker walks down the scored list and applies penalties for repeating a category or seller, inserts one or two exploration slots for new items, enforces sponsored placements and policy filters, and removes anything shown to this user in their last few sessions without a click. It is where business rules belong, so they never leak into model training.