Implement GELU Activation (Gaussian Error Linear Unit)
Implement GELU Activation (Gaussian Error Linear Unit)
Implement the Gaussian Error Linear Unit (GELU) activation function. It smoothly combines the behavior of linear and nonlinear activations by weighting each input x by the probability that a Gaussian random variable is less than x.
Mathematical Definition
GELU(x)=21x(1+erf(2x))Examples
Input:
x = [-1.0, 0.0, 1.0]
Output:
[-0.158655, 0.0, 0.841345]
Input:
x = [[-2., -1.],[0., 1.]]
Output:
[[-0.045500, -0.158655],
[ 0.000000, 0.841345]]
Hint 1
Convert input to NumPy array first.
Hint 2
Use np.vectorize(math.erf) to make the error function work with arrays.
Requirements
- Must handle scalars, lists, and NumPy arrays
- Must be fully vectorized (no loops)
- Return a NumPy array of floats
Constraints
- Input up to size 1e6
- Time limit: 200 ms; Memory: 64 MB
Try Similar Problems
Log in to take notes on this problem
Accepts: array
Implement GELU Activation (Gaussian Error Linear Unit)
Implement GELU Activation (Gaussian Error Linear Unit)
Implement the Gaussian Error Linear Unit (GELU) activation function. It smoothly combines the behavior of linear and nonlinear activations by weighting each input x by the probability that a Gaussian random variable is less than x.
Mathematical Definition
GELU(x)=21x(1+erf(2x))Examples
Input:
x = [-1.0, 0.0, 1.0]
Output:
[-0.158655, 0.0, 0.841345]
Input:
x = [[-2., -1.],[0., 1.]]
Output:
[[-0.045500, -0.158655],
[ 0.000000, 0.841345]]
Hint 1
Convert input to NumPy array first.
Hint 2
Use np.vectorize(math.erf) to make the error function work with arrays.
Requirements
- Must handle scalars, lists, and NumPy arrays
- Must be fully vectorized (no loops)
- Return a NumPy array of floats
Constraints
- Input up to size 1e6
- Time limit: 200 ms; Memory: 64 MB
Try Similar Problems
Log in to take notes on this problem
Accepts: array