Implement KL Divergence
Implement KL Divergence
Implement KL Divergence (Kullback-Leibler divergence) to measure how one probability distribution differs from another.
KL Divergence Formula:
DKL(P∥Q)=i∑PilogQiPiwhere P and Q are probability distributions
Function Arguments
p: array-like- First probability distribution, shape (N,)q: array-like- Second probability distribution, shape (N,)eps: float = 1e-12- Numerical stability epsilon
Examples
Input: p=[0.4, 0.6], q=[0.5, 0.5], eps=1e-12
Output: 0.0201
Small difference between similar distributions
Input: p=[0.3, 0.7], q=[0.3, 0.7], eps=1e-12
Output: 0.0
Identical distributions have KL divergence of 0
Input: p=[0.9, 0.1], q=[0.5, 0.5], eps=1e-12
Output: 0.368
Concentrated vs uniform distribution has higher divergence
Hint 1
Add eps to q for stability: q_stable = q + eps before computing log ratios.
Hint 2
Only compute terms where p[i] > 0, since 0 * log(0/q) = 0 by convention.
Hint 3
Use np.log() for element-wise logarithm and np.sum() for the final result.
Requirements
- Add eps to q to prevent log(0)
- Compute elementwise and sum the results
- Handle case where p[i] = 0 (contributes 0 to sum)
- Return scalar KL divergence
Constraints
- p, q ≥ 0 and sum to 1 (probability distributions)
- Works for N up to 10k elements
- NumPy only; time limit: 200ms
Try Similar Problems
Log in to take notes on this problem
Accepts: array
Accepts: array
Accepts: number
Implement KL Divergence
Implement KL Divergence
Implement KL Divergence (Kullback-Leibler divergence) to measure how one probability distribution differs from another.
KL Divergence Formula:
DKL(P∥Q)=i∑PilogQiPiwhere P and Q are probability distributions
Function Arguments
p: array-like- First probability distribution, shape (N,)q: array-like- Second probability distribution, shape (N,)eps: float = 1e-12- Numerical stability epsilon
Examples
Input: p=[0.4, 0.6], q=[0.5, 0.5], eps=1e-12
Output: 0.0201
Small difference between similar distributions
Input: p=[0.3, 0.7], q=[0.3, 0.7], eps=1e-12
Output: 0.0
Identical distributions have KL divergence of 0
Input: p=[0.9, 0.1], q=[0.5, 0.5], eps=1e-12
Output: 0.368
Concentrated vs uniform distribution has higher divergence
Hint 1
Add eps to q for stability: q_stable = q + eps before computing log ratios.
Hint 2
Only compute terms where p[i] > 0, since 0 * log(0/q) = 0 by convention.
Hint 3
Use np.log() for element-wise logarithm and np.sum() for the final result.
Requirements
- Add eps to q to prevent log(0)
- Compute elementwise and sum the results
- Handle case where p[i] = 0 (contributes 0 to sum)
- Return scalar KL divergence
Constraints
- p, q ≥ 0 and sum to 1 (probability distributions)
- Works for N up to 10k elements
- NumPy only; time limit: 200ms
Try Similar Problems
Log in to take notes on this problem
Accepts: array
Accepts: array
Accepts: number