Problems
Loading...
1 / 1

RMSProp Optimizer (Single Update Step)

Optimization
Easy

Implement one update step of the RMSProp optimizer. Given current parameters, gradients, and running squared gradient accumulator, return updated parameters and accumulator.

Step 1: Update Running Average

st=β⋅st−1+(1−β)⋅gt2s_t = \beta \cdot s_{t-1} + (1 - \beta) \cdot g_t^2

Step 2: Parameter Update

wt=wt−1−ηst+ε⋅gtw_t = w_{t-1} - \frac{\eta}{\sqrt{s_t + \varepsilon}} \cdot g_t

Where: w = parameters, g = gradients, s = squared gradient accumulator, η = learning rate, β = decay factor, ε = stability constant

Loading visualization...

Examples

Input: w=[1.0, 2.0], g=[0.2, -0.4], s=[0, 0], lr=0.1

Output: ([0.684, 2.316], [0.004, 0.016])

Input: w=[5.0], g=[0.0], s=[0.1], lr=0.1

Output: ([5.0], [0.09])

Hint 1

Convert inputs to NumPy arrays first. Update s using the exponential moving average formula, then use it to update w.

Hint 2

Use element-wise operations: g * g for squared gradients, np.sqrt() for square root, and standard arithmetic operators.

Requirements

  • Return tuple (new_w, new_s) with same shapes as inputs
  • Use the exact update formulas above
  • Vectorized implementation only (no Python loops)
  • Handle any array shape (1D, 2D, etc.)

Constraints

  • Parameter dimension D: 1 ≤ D ≤ 10⁵
  • Learning rate lr > 0
  • Decay factor: 0 < β < 1
  • Libraries: NumPy only
Try Similar Problems
Adam OptimizerNesterov MomentumNadam OptimizerAdagrad OptimizerAdamw Optimizer
PrevNext