X AlgorithmPortuguês
Back to overview

03

Two sources of posts

Now it gathers candidates from two places, at the same time. Posts from accounts you follow (“in-network”, served by Thunder), and posts from people you don’t follow but might enjoy (“out-of-network”, found by Phoenix).

Two scouts go out: one combs through your inner circle’s posts, the other searches the entire world for things that smell like you.

Under the hood: Thunder is a Rust in-memory store fed by Kafka, with sub-millisecond lookups. Phoenix retrieval uses a two-tower model: one tower turns you into a vector, another turns posts into vectors, and the closest matches win. thunder/ · phoenix/recsys_retrieval_model.py

A little deeper

Thunder ingests post create and delete events from Kafka and keeps recent posts in memory, organized per user. It can answer “what did the people I follow just post?” in under a millisecond.

Phoenix retrieval is the magic of the “For You” feed: it can surface posts from people you’ve never heard of. The user tower encodes your taste; the candidate tower encodes every post; a similarity search picks the best matches.

View on GitHubMade by Henrique Martins