Source
MAPS: Netflix’s Multimodal Asset Personalization at Scale
Netflix implemented multimodal embeddings—mathematical representations of images, text, and audio—to personalize artwork and video previews, solving the cold-start problem where new content lacks interaction data.
Why it matters
This allows Netflix to show users more relevant images and videos immediately after a title launches, improving discovery without waiting for large amounts of user interaction data to accumulate.
The details
- CLIP embeddings remain consistent across different image crops and resize ratios. - MediaFM was trained on 80 million shots to combine visual, audio, and text signals. - Inverse Propensity Scoring (IPS) is used to provide unbiased offline metric evaluations. - The Netflix Embedding Store decouples foundation-model updates from the deployment of personalization models.
What's next
Netflix aims to create a single shared semantic space for image, text, and video to enable cross-modal retrieval.
Show entities and relationshipsHide entities and relationships
In this article
Key connections
Netflix owns Netflix Embedding Store
The Embedding Store is a component of Netflix's AI Platform.
Netflix encodes each artwork with CLIP and folds the resulting embedding into its asset representation.
CLIP is related to Netflix Embedding Store
CLIP embeddings for artwork live in the Netflix Embedding Store.
SeqCLIP is related to Netflix Embedding Store
SeqCLIP embeddings for video previews live in the Netflix Embedding Store.
MediaFM is related to Netflix Embedding Store
MediaFM embeddings for video previews live in the Netflix Embedding Store.
Netflix uses Inverse Propensity Scoring
Netflix computes inverse propensity scoring on a dedicated slice of exploration traffic to get unbiased offline evaluations.
Show 8 more connectionsShow fewer connections
Netflix uses Linear Probe
Netflix trains a linear probe on candidate embeddings to cheaply screen them before full offline and online testing.
MediaFM fuses SeqCLIP as its visual signal, alongside audio and text embeddings, into a single per-shot embedding.
Aneesh Vartakavi works at Netflix
Credited for the CLIP embedding and MediaFM work described in the article.
Santiago Castro works at Netflix
Credited for the CLIP embedding and MediaFM work described in the article.
Avneesh Saluja works at Netflix
Credited for the CLIP embedding and MediaFM work described in the article.
Ratna Kavuri works at Netflix
Credited for the backend systems that serve multimedia personalization in production.
MediaFM is Netflix's first in-house multimodal foundation model.
SeqCLIP was Netflix's first content-aware video preview embedding.
Related events
Netflix Ships Multimodal Embeddings for Cold-Start Asset Personalization
Get the weekly recap
The stories like this one, picked and explained — once a week, straight to your inbox.