Problems

1 / 1

Detect Train-Serving Skew

MLOps

Medium

Detect train-serving skew by computing the Population Stability Index (PSI) for each feature. Given training and serving feature distributions as dictionaries mapping feature names to lists of bin proportions, compute PSI per feature and flag which features have shifted beyond a given threshold.

Mathematical Definition

PSI = \sum_{i=1}^{n} (P_i^{serving} - P_i^{train}) \times \ln\left(\frac{P_i^{serving}}{P_i^{train}}\right)

A feature is flagged as skewed when its PSI ≥ threshold.

Loading visualization...

Examples

Input:

train_dist = {"age": [0.1, 0.2, 0.3, 0.25, 0.15], "income": [0.2, 0.2, 0.2, 0.2, 0.2]} serving_dist = {"age": [0.05, 0.1, 0.15, 0.35, 0.35], "income": [0.2, 0.2, 0.2, 0.2, 0.2]} threshold = 0.2

Output:

{"age": {"psi": 0.4111, "skewed": True}, "income": {"psi": 0.0, "skewed": False}}

The "age" distribution has shifted significantly (PSI ≈ 0.41 ≥ 0.2), while "income" remains identical.

Input:

train_dist = {"clicks": [0.3, 0.4, 0.2, 0.1]} serving_dist = {"clicks": [0.25, 0.35, 0.25, 0.15]} threshold = 0.2

Output:

{"clicks": {"psi": 0.0472, "skewed": False}}

A gentle shift yields PSI ≈ 0.047, well below the 0.2 threshold - no skew flagged.

Hint 1

A small constant added to each bin prevents log(0) and division by zero.

Hint 2

Each PSI term measures how much a single bin has shifted between the two distributions.

Hint 3

NumPy arrays and vectorized operations can replace explicit Python loops over bins.

Requirements

Compute PSI for each feature independently
Use the provided eps to prevent numerical errors with zero-probability bins
Return a dict mapping each feature name to {"psi": float, "skewed": bool}
Use NumPy only

Constraints

Number of features ≤ 100
Bins per feature ≤ 1000
All bin values are non-negative floats
Use NumPy only
Time limit: 300 ms

Try Similar Problems

Data Drift Detection Monitoring Metrics Selection Shadow Deployment Evaluation Feature Store Lookup Retraining Trigger Design

Prev Next

Detect Train-Serving Skew

MLOps

Medium

Mathematical Definition

PSI = \sum_{i=1}^{n} (P_i^{serving} - P_i^{train}) \times \ln\left(\frac{P_i^{serving}}{P_i^{train}}\right)

A feature is flagged as skewed when its PSI ≥ threshold.

Loading visualization...

Examples

Input:

train_dist = {"age": [0.1, 0.2, 0.3, 0.25, 0.15], "income": [0.2, 0.2, 0.2, 0.2, 0.2]} serving_dist = {"age": [0.05, 0.1, 0.15, 0.35, 0.35], "income": [0.2, 0.2, 0.2, 0.2, 0.2]} threshold = 0.2

Output:

{"age": {"psi": 0.4111, "skewed": True}, "income": {"psi": 0.0, "skewed": False}}

The "age" distribution has shifted significantly (PSI ≈ 0.41 ≥ 0.2), while "income" remains identical.

Input:

train_dist = {"clicks": [0.3, 0.4, 0.2, 0.1]} serving_dist = {"clicks": [0.25, 0.35, 0.25, 0.15]} threshold = 0.2

Output:

{"clicks": {"psi": 0.0472, "skewed": False}}

A gentle shift yields PSI ≈ 0.047, well below the 0.2 threshold - no skew flagged.

Hint 1

A small constant added to each bin prevents log(0) and division by zero.

Hint 2

Each PSI term measures how much a single bin has shifted between the two distributions.

Hint 3

NumPy arrays and vectorized operations can replace explicit Python loops over bins.

Requirements

Compute PSI for each feature independently
Use the provided eps to prevent numerical errors with zero-probability bins
Return a dict mapping each feature name to {"psi": float, "skewed": bool}
Use NumPy only

Constraints

Number of features ≤ 100
Bins per feature ≤ 1000
All bin values are non-negative floats
Use NumPy only
Time limit: 300 ms

Try Similar Problems

Data Drift Detection Monitoring Metrics Selection Shadow Deployment Evaluation Feature Store Lookup Retraining Trigger Design

Case 1

Case 2

train_dist =

Accepts: any

serving_dist =

Accepts: any

threshold =

Accepts: number

eps =

Accepts: number