Recommendation systems: retrieval, ranking & cold start

Thirty slots, eight million tracks

Lena opens a music app on a Monday morning. Near the top of the home screen is a row called "For you" with room for 30 tracks, and the catalogue holds 8 million. In two years she has played about 600 tracks, mostly Brazilian jazz and some 90s hip-hop. The row has to be hers, and the recommender's share of the page's time is around 100 milliseconds.

Score everything, wait eighty seconds

The direct design is one model that estimates how likely Lena is to play a given track through, run over every track, keeping the best 30. Say the model needs 10 microseconds per track on one CPU core. Eight million tracks make 80 seconds of compute for one home screen. A hundredfold speed-up from batching or GPUs leaves 0.8 seconds, still eight times the budget, for one listener.

Narrow cheaply, then rank carefully

Almost all of those scores are wasted, so recommenders split the work. Retrieval, also called candidate generation, uses cheap methods to pull a few hundred plausible tracks out of millions. Ranking runs the expensive model on those survivors only. Re-ranking then applies rules that no model score expresses well. One request for Lena's row goes like this, with sizes that are typical, not fixed:

1. The recommender reads Lena's profile vector and her last 20 plays from a fast store.
2. Three sources run in parallel, each in a few milliseconds: the 400 tracks nearest to her vector, 300 tracks often played by people who played her recent ones, and 100 tracks trending in her country.
3. The recommender merges the 800, drops duplicates, tracks she skipped this week and tracks not licensed in her country, and keeps about 600.
4. The ranking model scores all 600 with richer inputs, such as her history with the artist and what she played a minute ago: at 10 microseconds a track, about 6 ms of compute.
5. The re-ranker walks down the scored list and fills the row: at most two tracks per artist, two slots kept for new releases among the candidates, 30 tracks in all.
6. The recommender returns the row and logs which 30 tracks it showed, and in which positions.

Retrieval can afford rough scores, since ranking reorders whatever it returns, but not misses: a track it leaves out never reaches the ranking model in this request. That is why retrieval draws on several sources, each missing different things. Re-ranking holds what the product wants regardless of score: eight tracks by one artist may each score highest and still make a poor row.

Two ways to find what she might like

The sources differ in what they know. Collaborative filtering works from interactions alone: listeners whose histories overlap Lena's played this track, or people who played the track she just finished also played that one. In a graph linking listeners to the tracks they played, that second question is a two-hop traversal (Graph databases). Collaborative filtering knows nothing about the track itself. That lets it connect Brazilian jazz to a city-pop record no tag would link, and it is also the weakness: a track nobody has played does not exist for it, and a hugely popular track co-occurs with almost everything.

Content-based recommendation compares the items themselves: genre, tempo, the audio, the artist, the language of the lyrics. It works for a track released an hour ago, because the attributes exist before the plays do. Its failure runs the other way: left alone, it finds more of what Lena already plays, and her row drifts toward one genre. Most production systems use both, as separate retrieval sources and as inputs to the ranking model.

A common way to put collaborative filtering into retrieval is to learn an embedding, a vector of tens to hundreds of numbers, for every listener and every track, trained so that a listener's vector lies close to the tracks they play. Matrix factorisation learns them from plays alone; a two-tower model computes them with one neural network per side, which can also take content features as input. Retrieval becomes a nearest-neighbour search for the 400 track vectors closest to Lena's, usually on an approximate index that trades a few missed neighbours for speed; Vector search: embeddings, nearest neighbours & hybrid search explains those indexes.

What a play does and does not say

Lena never rates anything. The system learns from implicit feedback: plays, early skips, saves, tracks played to the end. A play shows that the track started, not that she liked it, so a model trained only to maximise plays can learn to favour tracks that are easy to start and quick to skip. A missing play says even less: usually she never saw the track, which is why step 6 logs what was shown.

The two halves of the system run on different clocks: a training job on a schedule, often nightly, learns the embeddings and the ranking model from months of logged events, while a stream processor keeps signals such as the last five minutes of skips fresh for the request path. Serving the ranking model inside the budget and rolling out a new version safely is Model serving: inference, versions & safe rollout.

No history, no plays

Lena's flatmate signs up the same morning. He has no plays, so matrix factorisation has no vector for him, and a two-tower model can compute one only from what it does have; his first row has to come from those other signals: what is popular in his country, the three artists he picked during sign-up, his device, the time of day. That is the cold start problem for a new user, and it eases as he plays: sources that read the current session can use those plays at once, and his vector reflects them when the system next recomputes it.

A track released that morning has the reverse problem. Collaborative sources cannot see it until people play it, and few people find a track that is never shown. Content-based sources can place it on day one from its audio and metadata, and so can a two-tower model whose track side takes content as input. Many systems also reserve a small share of slots for exploration: new or rarely shown tracks offered to listeners whose taste suggests a fit, so the system collects the interactions it lacks.

It learns from what it showed

A recommender's training data is made by the recommender. Tracks it showed had the chance to be played and tracks it never showed had far fewer, so a model trained on those plays without correcting for that can learn that the shown ones are what people like. Over time this feedback loop can narrow the row toward what was already popular, a popularity bias that squeezes out new artists. No single fix removes it; the usual counters are exploration, logging impressions so a model can tell "not shown" from "shown and ignored", and correcting for position, since a track in slot 1 gets played more partly because it is in slot 1.

The same loop limits offline evaluation: last week's held-out plays happened under the old model, which chose what people could see, so a new model that ranks them higher does not always do better with real listeners. Ranking models are compared live with an A/B test, which A/B testing & experimentation explains.

Lena's row on a Grid

Recommendation Engine asks for this system in up to 18 nodes. Draw retrieval and ranking as separate stages with the candidate count on the edge between them, give retrieval at least one source fed by interactions and one by content, and put the event path beside it: interactions into Kafka, a stream processor for fresh signals, object storage for training. Mark where the flatmate's first row comes from, and how the new track earns its first plays.

One part of that Grid stays a single arrow here: the features the ranking model reads, such as an artist's plays in the last 7 days, must be computed the same way in training as at request time, or the model meets different numbers in production. Feature stores: online, offline & training-serving skew takes that arrow apart.