Recommendation Systems

Recommendation Systems

Purpose

Design and build industrial-grade recommendation systems using the multi-stage architecture adopted by Alibaba, ByteDance, Google, Meta, and other top tech companies.

How It Works

System Architecture (Industrial Standard)

┌──────────────────────────────────────────────────────┐
│                    User Request                       │
├──────────────────────────────────────────────────────┤
│  Stage 1: RECALL (Candidate Generation)               │
│  Input: billions of items → Output: ~1000 candidates  │
│  Models: DSSM, SASRec, MIND, Swing, ItemCF, ANN      │
│  Latency budget:  50. Use MMOE for CTR + CVR multi-task.

"Our recommendation system has a 'filter bubble' problem —
users keep seeing the same categories"
→ Add DPP diversity re-ranking, exploration channels in recall,
  category-aware negative sampling, position-debiased training

Output Format

• System Architecture: Multi-stage pipeline with model choices per stage
• Model Code: PyTorch implementation with training loop
• Feature Schema: Feature list with types and encoding strategy
• Offline Evaluation: AUC, GAUC, NDCG, HitRate@K metrics
• Online Metrics: CTR, CVR, GMV, diversity, coverage, novelty
• Serving Design: Latency budget, ANN index, feature store

---

Key References

• Zhou et al. (2018) — DIN: Deep Interest Network for Click-Through Rate Prediction (Alibaba)
• Zhou et al. (2019) — DIEN: Deep Interest Evolution Network (Alibaba)
• Chen et al. (2019) — BST: Behavior Sequence Transformer for E-commerce Recommendation (Alibaba)
• Wang et al. (2021) — DCN V2: Improved Deep & Cross Network (Google)
• Ma et al. (2018) — MMOE: Modeling Task Relationships in Multi-Task Learning (Google)
• Tang et al. (2020) — PLE: Progressive Layered Extraction for Multi-Task Learning (Tencent)
• Huang et al. (2013) — DSSM: Learning Deep Structured Semantic Models (Microsoft)
• Kang & McAuley (2018) — SASRec: Self-Attentive Sequential Recommendation (UCSD)
• Pi et al. (2020) — SIM: Search-based Interest Model (Alibaba)
• Guo et al. (2017) — DeepFM: A Factorization-Machine based Neural Network (Huawei)

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ericwang915
• Source: ericwang915/data-scientist-skills
• License: MIT

Install and usage instructions live in the source repository linked above.