Skip to content

DOCUMENTATION.md

Latest commit

 

History

History
339 lines (245 loc) · 9.8 KB

DOCUMENTATION.md

File metadata and controls

339 lines (245 loc) · 9.8 KB
Raw

LTR Documentation

API Endpoints

Query Endpoint

GET /query

Samples a random user and query from the loaded dataset.

Response:

{
    "user_id": "8a2f5b18-1a23-4c72-8de1-9c7d4a89ff01",
    "query_id": "7b9f3a12-6d15-4c9f-92da-348ab2a83b10",
    "query_text": "economic recovery 2025"
}

Usage:

curl http://localhost:3000/query

Ranklist Endpoint

POST /ranklist

Simulates user actions for a ranked list of articles given a query ID and user ID. There are 200 different users in the simulation.

Request Body:

{
    "query_id": "7b9f3a12-6d15-4c9f-92da-348ab2a83b10",
    "user_id": "8a2f5b18-1a23-4c72-8de1-9c7d4a89ff01",
    "ranked_article_ids": [
        "550e8400-e29b-41d4-a716-446655440000",
        "550e8400-e29b-41d4-a716-446655440001",
        "550e8400-e29b-41d4-a716-446655440002"
    ]
}

Response:

{
    "actions": [
        [
            "Click",
            {"Dwell": {"secs": 45, "nanos": 123000000}},
            "Like"
        ],
        [],
        [
            "Click",
            {"Dwell": {"secs": 20, "nanos": 456000000}}
        ]
    ]
}

Response Format:

  • actions: Array of action sequences, one for each article in the ranked list
  • Empty array [] indicates the user skipped that article
  • Non-empty arrays contain ordered user actions:
    • "Click": User clicked on the article
    • {"Dwell": {...}}: Time spent reading (follows Click)
    • "Like": User liked the article (optional)
    • "Share": User shared the article (optional)
    • "Bookmark": User bookmarked the article (optional)

Learning-to-Rank Models

Logistic Regression (Pointwise LTR)

Files: code/logistic_regression/ Config: configs/logistic_regression_config.yaml

A pointwise Learning-to-Rank system that predicts click probability for user-article pairs using logistic regression.

Training Phase (train.py)

Data Preparation:

  • Processes historical interaction logs from baseline_logs_path
  • Creates binary labels: 1 for clicked articles, 0 for skipped articles
  • Combines article features with user profile vectors
  • Outputs structured dataset with features X and labels y

Model Training:

  • Splits data into 80% training and 20% testing sets
  • Applies StandardScaler normalization for feature scaling
  • Trains LogisticRegression with class_weight="balanced" to handle click imbalance
  • Saves trained model and scaler using joblib

Inference Phase (test.py)

Setup:

  • Loads pre-trained model and scaler
  • Connects to Elasticsearch for candidate retrieval

Ranking Process:

  1. Retrieves top N candidates from Elasticsearch based on query text
  2. Handles cold start users by falling back to Elasticsearch ranking
  3. Constructs feature vectors: [Article_Features + User_Features]
  4. Applies scaling and predicts click probabilities
  5. Ranks articles by predicted probability (highest first)

Simulation Loop:

  • Fetches user queries and generates personalized rankings
  • Posts rankings to simulator and logs user interactions
  • Creates feedback loop for model retraining

Reinforcement Learning (Online LTR)

Files: code/reinforcement_learning/ Config: configs/rl_config.yaml

An online Learning-to-Rank system using contextual bandits with epsilon-greedy exploration strategy.

Model Architecture (model.py)

RLAgent Class:

  • curr_version: Model iteration number
  • explore_rate: Epsilon value for exploration probability
  • model: Random Forest classifier for decision making

Mathematical Framework:

State Representation: Feature vector construction for user u and document d: $$x = [V_u, V_d, V_u \odot V_d]$$ Where V_u is user preference vector, V_d is document attributes, and ⊙ is element-wise multiplication.

Epsilon-Greedy Policy: $$\text{Action} = \begin{cases} \text{Random Shuffle} & \text{with probability } \epsilon \ \text{Argmax } P(\text{click}|x) & \text{with probability } 1 - \epsilon \end{cases}$$

Scoring: $$\text{Score}(d) = P(y=1 | x; \theta)$$

Training Pipeline (train.py)

Iterative Learning Process:

  1. Bootstrap (v₀ → v₁): Train initial model on baseline logs
  2. Iterative Collection: For each version k:
    • Load model vₖ
    • Interact with simulator using epsilon-greedy policy
    • Log interactions to intermediate logs
  3. Aggregated Training: Train vₖ₊₁ on combined data: $$\mathcal{D}{train} = \mathcal{D}{baseline} \cup \mathcal{D}{iter_1} \cup \ldots \cup \mathcal{D}{iter_k}$$

Binary Classification:

  • Target y=1: User clicked the article
  • Target y=0: Article shown but not clicked

Testing Pipeline (test.py)

Pure Exploitation Mode:

  • Disables exploration (ε=0) for final evaluation
  • Loads specific model version and feature definitions
  • Handles cold start users with Elasticsearch fallback
  • Ranks by predicted click probability
  • Logs results for offline analysis

RankNet (Pairwise LTR)

Files: code/rank_net/ Config: configs/rank_net_config.yaml Paper: RankNet: A Ranking Approach to Learning

A pairwise Learning-to-Rank model using neural networks to minimize probabilistic ranking loss.

Model Architecture (model.py)

Neural Network Configuration:

  • input_dim: Feature dimension (dynamic + static + user features)
  • hidden_dims: List of hidden layer sizes
  • activation: Activation function (tanh, relu, sigmoid, gelu)

Mathematical Foundation:

Pairwise Probability: For documents i and j with scores sᵢ and sⱼ: $$P_{ij} = \frac{1}{1 + e^{-\sigma (s_i - s_j)}}$$

Cost Function (RankNet Loss): $$C = \log(1 + e^{-\sigma(s_i - s_j)})$$

Training Pipeline (train.py)

Relevance Scoring: Combines explicit feedback with dwell time: $$\text{Score} = \sum_{a \in A} w(a) + \log(1 + t)$$

Action Weights:

  • Click: +1.0
  • Like: +2.0
  • Bookmark: +3.0

Pair Construction:

  • relevance(doc_a) > relevance(doc_b) → target_p = 1.0
  • relevance(doc_a) == relevance(doc_b) → target_p = 0.5

Testing Pipeline (test.py)

Inference Process:

  1. Load PyTorch model, scalers, and feature stores
  2. Retrieve candidate documents from Elasticsearch
  3. Construct feature vectors (dynamic + static + user)
  4. Predict scalar scores for each document
  5. Rank documents by predicted scores (descending)

LambdaMART (Listwise LTR)

Files: code/lambda_mart/ Config: configs/lambda_mart_config.yaml Paper: From RankNet to LambdaRank to LambdaMART

A boosted tree implementation of LambdaRank that optimizes information retrieval metrics directly.

Model Architecture (model.py)

LambdaMART Parameters:

  • n_trees: Number of boosting rounds
  • learning_rate: Scaling factor for tree predictions
  • max_leaf_nodes: Maximum leaves per tree
  • sigma: Sigmoid parameter for probability curve

Mathematical Framework:

Lambda Gradients: $$\lambda_{ij} = \frac{-\sigma |\Delta Z_{ij}|}{1 + e^{\sigma(s_i - s_j)}}$$

Hessian Calculation: $$\frac{\partial^2 C}{\partial s_i^2} = \sigma^2 |\Delta Z_{ij}| \rho_{ij} (1 - \rho_{ij})$$

Newton Step (Leaf Values): $$\gamma_{km} = \frac{\sum_{x \in R_{km}} \lambda_i}{\sum_{x \in R_{km}} \text{Hessian}_i}$$

Training Pipeline (train.py)

Query Group Processing:

  • Maps string Query IDs to integer qids for grouping
  • Constructs feature vectors: [Dynamic, Article, User Features]
  • Uses relevance scores to compute ideal NDCG
  • Iteratively adds trees to minimize listwise cost

Testing Pipeline (test.py)

Ensemble Prediction: $$F_N(x) = \sum_{i=1}^N \alpha_i f_i(x)$$

Inference Steps:

  1. Load ensemble model and feature stores
  2. Retrieve Elasticsearch candidates
  3. Construct feature matrices with consistent ordering
  4. Predict scores using tree ensemble
  5. Rank documents by ensemble scores

A/B Testing Framework

Evaluates personalized ranking models against baseline Elasticsearch using controlled experiments.

Experimental Design

Process Flow:

  1. Fetch (user_id, query_text) from simulator
  2. Assign user to experiment group via GrowthBook
  3. Generate rankings:
    • Control: Elasticsearch baseline
    • Variant: ML model inference
  4. Send rankings to simulator and collect user actions
  5. Log interactions with dynamic features
  6. Update click-through rate and dwell time statistics

GrowthBook Integration

GrowthBookManager Class:

  • Implements deterministic user assignment using consistent hashing
  • Maintains 50-50 weighted split between control and variant
  • Ensures stable assignment across user sessions
  • Tracks experiment metrics for statistical analysis

Feature Configuration:

  • Feature: "personalized-ranking"
  • Variations: False (control), True (variant)
  • Hash attribute: User ID
  • Salt: Prevents experiment collision

Tracked Metrics:

{
    "control": {"impressions": 0, "clicks": 0, "dwell_time": 0.0},
    "variant": {"impressions": 0, "clicks": 0, "dwell_time": 0.0}
}

System Features

Document Processing

  • Multi-language support: Configurable language detection and processing
  • Custom stopwords: Language-specific stopword removal
  • Feature extraction: BM25 score, title length, text length

Performance Optimization

  • Cold start handling: Fallback to Elasticsearch for new users
  • Feature caching: Efficient storage and retrieval of computed features
  • Batch processing: Optimized handling of multiple queries

Evaluation Metrics

  • Click-through Rate (CTR): Percentage of clicked results
  • Dwell Time: Time spent reading articles
  • NDCG: Normalized Discounted Cumulative Gain
  • Statistical Significance: A/B test result validation