Label Smoothing Loss
Label Smoothing Loss
Label smoothing is a regularization technique that prevents a model from becoming overconfident. Instead of using hard one-hot targets (1 for the correct class, 0 for all others), it softens the target distribution by redistributing a small fraction of probability mass to the incorrect classes.
Given a predicted probability distribution, a target class index, and a smoothing parameter epsilon, compute the cross-entropy loss with smoothed labels.
Algorithm
- Build the smoothed target distribution. For K classes and smoothing factor epsilon:
- Compute the cross-entropy loss between the smoothed targets q and the predictions p:
Examples
Input:
predictions = [0.9, 0.05, 0.05], target = 0, epsilon = 0.1
Output:
0.3984
K = 3. Smoothed targets: q = [0.9333, 0.0333, 0.0333]. Loss = -0.9333 * ln(0.9) - 0.0333 * ln(0.05) - 0.0333 * ln(0.05).
Input:
predictions = [0.7, 0.3], target = 0, epsilon = 0.2
Output:
0.4413
K = 2. Smoothed targets: q = [0.9, 0.1]. Loss = -0.9 * ln(0.7) - 0.1 * ln(0.3).
Hint 1
First compute K = len(predictions). For the target class, the smoothed label is (1 - epsilon + epsilon/K). For all other classes it is epsilon/K.
Hint 2
Use math.log() for the natural logarithm. Loop over all classes, multiply the smoothed label by log(prediction), and sum the negated values.
Requirements
- Build the smoothed target distribution using the formula above
- Compute cross-entropy between smoothed targets and predictions
- K (number of classes) is the length of the predictions list
- Return a single float
Constraints
- predictions contains positive probabilities that sum to approximately 1
- 0 <= epsilon <= 1
- 0 <= target < len(predictions)
- Return a single float
- Time limit: 300 ms
Try Similar Problems
Log in to take notes on this problem
Accepts: array
Accepts: number
Accepts: number
Label Smoothing Loss
Label Smoothing Loss
Label smoothing is a regularization technique that prevents a model from becoming overconfident. Instead of using hard one-hot targets (1 for the correct class, 0 for all others), it softens the target distribution by redistributing a small fraction of probability mass to the incorrect classes.
Given a predicted probability distribution, a target class index, and a smoothing parameter epsilon, compute the cross-entropy loss with smoothed labels.
Algorithm
- Build the smoothed target distribution. For K classes and smoothing factor epsilon:
- Compute the cross-entropy loss between the smoothed targets q and the predictions p:
Examples
Input:
predictions = [0.9, 0.05, 0.05], target = 0, epsilon = 0.1
Output:
0.3984
K = 3. Smoothed targets: q = [0.9333, 0.0333, 0.0333]. Loss = -0.9333 * ln(0.9) - 0.0333 * ln(0.05) - 0.0333 * ln(0.05).
Input:
predictions = [0.7, 0.3], target = 0, epsilon = 0.2
Output:
0.4413
K = 2. Smoothed targets: q = [0.9, 0.1]. Loss = -0.9 * ln(0.7) - 0.1 * ln(0.3).
Hint 1
First compute K = len(predictions). For the target class, the smoothed label is (1 - epsilon + epsilon/K). For all other classes it is epsilon/K.
Hint 2
Use math.log() for the natural logarithm. Loop over all classes, multiply the smoothed label by log(prediction), and sum the negated values.
Requirements
- Build the smoothed target distribution using the formula above
- Compute cross-entropy between smoothed targets and predictions
- K (number of classes) is the length of the predictions list
- Return a single float
Constraints
- predictions contains positive probabilities that sum to approximately 1
- 0 <= epsilon <= 1
- 0 <= target < len(predictions)
- Return a single float
- Time limit: 300 ms
Try Similar Problems
Log in to take notes on this problem
Accepts: array
Accepts: number
Accepts: number