Statements (35)
| Predicate | Object |
|---|---|
| gptkbp:instance_of |
gptkb:microprocessor
|
| gptkbp:analyzes |
a line with a small slope for negative inputs
|
| gptkbp:can_be |
different values for alpha
|
| gptkbp:can_provide |
gradient flow
|
| gptkbp:defines |
f(x) = x if x > 0 else alpha * x
|
| gptkbp:enhances |
model performance
|
| gptkbp:function |
allows a small, non-zero gradient when the unit is not active.
|
| gptkbp:has_method |
alpha
|
| https://www.w3.org/2000/01/rdf-schema#label |
Leaky Re LU
|
| gptkbp:introduced |
gptkb:Kaiming_He
|
| gptkbp:is_compared_to |
outliers than other functions
|
| gptkbp:is_considered |
activation in residual networks
|
| gptkbp:is_different_from |
everywhere
|
| gptkbp:is_effective_against |
deep architectures
dying Re LU problem |
| gptkbp:is_implemented_in |
gptkb:Graphics_Processing_Unit
gptkb:Py_Torch |
| gptkbp:is_often_compared_to |
saturate than sigmoid
|
| gptkbp:is_often_used_in |
deep learning models
|
| gptkbp:is_popular_in |
hidden layers
|
| gptkbp:is_scalable |
gptkb:microprocessor
efficient |
| gptkbp:is_similar_to |
Parametric Re LU
|
| gptkbp:is_used_in |
gptkb:microprocessor
gptkb:generative_adversarial_networks reinforcement learning transfer learning convolutional neural networks image classification tasks autoencoders natural language processing tasks |
| gptkbp:modifications |
linear activation function
|
| gptkbp:suitable_for |
standard Re LU in some cases
|
| gptkbp:type_of |
Re LU
|
| gptkbp:variant |
Rectified Linear Unit
|