Xavier Initialization
Xavier Initialization
Xavier (Glorot) initialization sets initial weights to maintain roughly the same variance of activations and gradients across layers. This prevents the vanishing or exploding gradient problem in networks using sigmoid or tanh activations.
Given a raw weight matrix W with values in [0, 1], fan_in (number of input units), and fan_out (number of output units), scale the weights to Xavier uniform initialization.
Algorithm
- Compute the Xavier uniform bound:
- Map each raw weight from [0, 1] to [-L, L]:
Examples
Input:
W = [[0.5, 0.5], [0.5, 0.5]], fan_in = 2, fan_out = 2
Output:
[[0.0, 0.0], [0.0, 0.0]]
With fan_in + fan_out = 4, limit = sqrt(6/4) = sqrt(1.5) = 1.2247. Raw value 0.5 maps to 0.5 * 2 * 1.2247 - 1.2247 = 0.
Input:
W = [[0, 1], [1, 0]], fan_in = 2, fan_out = 2
Output:
[[-1.2247, 1.2247], [1.2247, -1.2247]]
Raw 0 maps to -limit and raw 1 maps to +limit. The range [-1.2247, 1.2247] ensures proper variance for a layer with 2 inputs and 2 outputs.
Hint 1
Compute limit = sqrt(6 / (fan_in + fan_out)). Then for each element: scaled = raw * 2 * limit - limit. This linearly maps [0, 1] to [-limit, limit].
Hint 2
The mapping formula comes from: a value v in [0, 1] maps to v * (upper - lower) + lower. With lower = -limit and upper = limit, this gives v * 2 * limit - limit.
Requirements
- Scale raw uniform [0, 1] weights to Xavier uniform range [-limit, limit]
- Use the Xavier uniform formula: limit = sqrt(6 / (fan_in + fan_out))
- Return the scaled weight matrix as a list of lists of floats
Constraints
- W has at least 1 row and 1 column with values in [0, 1]
- fan_in >= 1, fan_out >= 1
- Return a list of lists of floats with the same shape as W
- Time limit: 300 ms
Log in to take notes on this problem
Accepts: array
Accepts: number
Accepts: number
Xavier Initialization
Xavier Initialization
Xavier (Glorot) initialization sets initial weights to maintain roughly the same variance of activations and gradients across layers. This prevents the vanishing or exploding gradient problem in networks using sigmoid or tanh activations.
Given a raw weight matrix W with values in [0, 1], fan_in (number of input units), and fan_out (number of output units), scale the weights to Xavier uniform initialization.
Algorithm
- Compute the Xavier uniform bound:
- Map each raw weight from [0, 1] to [-L, L]:
Examples
Input:
W = [[0.5, 0.5], [0.5, 0.5]], fan_in = 2, fan_out = 2
Output:
[[0.0, 0.0], [0.0, 0.0]]
With fan_in + fan_out = 4, limit = sqrt(6/4) = sqrt(1.5) = 1.2247. Raw value 0.5 maps to 0.5 * 2 * 1.2247 - 1.2247 = 0.
Input:
W = [[0, 1], [1, 0]], fan_in = 2, fan_out = 2
Output:
[[-1.2247, 1.2247], [1.2247, -1.2247]]
Raw 0 maps to -limit and raw 1 maps to +limit. The range [-1.2247, 1.2247] ensures proper variance for a layer with 2 inputs and 2 outputs.
Hint 1
Compute limit = sqrt(6 / (fan_in + fan_out)). Then for each element: scaled = raw * 2 * limit - limit. This linearly maps [0, 1] to [-limit, limit].
Hint 2
The mapping formula comes from: a value v in [0, 1] maps to v * (upper - lower) + lower. With lower = -limit and upper = limit, this gives v * 2 * limit - limit.
Requirements
- Scale raw uniform [0, 1] weights to Xavier uniform range [-limit, limit]
- Use the Xavier uniform formula: limit = sqrt(6 / (fan_in + fan_out))
- Return the scaled weight matrix as a list of lists of floats
Constraints
- W has at least 1 row and 1 column with values in [0, 1]
- fan_in >= 1, fan_out >= 1
- Return a list of lists of floats with the same shape as W
- Time limit: 300 ms
Log in to take notes on this problem
Accepts: array
Accepts: number
Accepts: number