Learning Rate Scheduler (Linear Decay)
Learning Rate Scheduler (Linear Decay)
Implement a linear learning rate schedule with optional warmup.
The learning rate starts at 0, increases linearly to an initial rate (η₀) over the warmup steps, then decays linearly toward a final rate (ηf) until total training steps are completed. After the total steps, the learning rate remains fixed at ηf.
Mathematical Definition
LR(t)=⎩⎨⎧Wt⋅η0ηf+(η0−ηf)⋅T−WT−tηfif t<Wif W≤t≤Tif t>Twhere t: current step (0-based), W: warmup steps, T: total steps, η₀: initial learning rate, ηf: final learning rate.
Examples
Input: step=0, total_steps=100, initial_lr=1e-3, final_lr=0.0, warmup_steps=10
Output: 0.0
Input: step=10, total_steps=100, initial_lr=1e-3, final_lr=0.0, warmup_steps=10
Output: 0.001
Input: step=50, total_steps=100, initial_lr=1e-3, final_lr=0.0, warmup_steps=10
Output: 0.00055
Hint 1
Handle each phase separately with conditional statements based on the current step.
Hint 2
Use linear interpolation between start and end values for both warmup and decay phases.
Constraints
- Scalar computation (no vectorization required)
- Must handle zero warmup and steps beyond total_steps
- Return a single float value
- Time limit: 100 ms
Try Similar Problems
Log in to take notes on this problem
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Learning Rate Scheduler (Linear Decay)
Learning Rate Scheduler (Linear Decay)
Implement a linear learning rate schedule with optional warmup.
The learning rate starts at 0, increases linearly to an initial rate (η₀) over the warmup steps, then decays linearly toward a final rate (ηf) until total training steps are completed. After the total steps, the learning rate remains fixed at ηf.
Mathematical Definition
LR(t)=⎩⎨⎧Wt⋅η0ηf+(η0−ηf)⋅T−WT−tηfif t<Wif W≤t≤Tif t>Twhere t: current step (0-based), W: warmup steps, T: total steps, η₀: initial learning rate, ηf: final learning rate.
Examples
Input: step=0, total_steps=100, initial_lr=1e-3, final_lr=0.0, warmup_steps=10
Output: 0.0
Input: step=10, total_steps=100, initial_lr=1e-3, final_lr=0.0, warmup_steps=10
Output: 0.001
Input: step=50, total_steps=100, initial_lr=1e-3, final_lr=0.0, warmup_steps=10
Output: 0.00055
Hint 1
Handle each phase separately with conditional statements based on the current step.
Hint 2
Use linear interpolation between start and end values for both warmup and decay phases.
Constraints
- Scalar computation (no vectorization required)
- Must handle zero warmup and steps beyond total_steps
- Return a single float value
- Time limit: 100 ms
Try Similar Problems
Log in to take notes on this problem
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number