|
gptkbp:instance_of
|
gptkb:neural_networks
|
|
gptkbp:analyzes
|
a line with a small slope for negative inputs
|
|
gptkbp:can
|
different values for alpha
|
|
gptkbp:can_provide
|
gradient flow
|
|
gptkbp:enhances
|
model performance
|
|
gptkbp:function
|
allows a small, non-zero gradient when the unit is not active.
|
|
gptkbp:has_a_parameter
|
alpha
|
|
gptkbp:helps_to_prevent
|
dying Re LU problem
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
Leaky Re LU
|
|
gptkbp:introduced_in
|
gptkb:Kaiming_He
|
|
gptkbp:is_a_choice_for
|
activation in residual networks
|
|
gptkbp:is_a_non-linear
|
gptkb:neural_networks
|
|
gptkbp:is_computationally
|
efficient
|
|
gptkbp:is_defined_by
|
f(x) = x if x > 0 else alpha * x
|
|
gptkbp:is_differentiable
|
everywhere
|
|
gptkbp:is_effective_against
|
deep architectures
|
|
gptkbp:is_implemented_in
|
gptkb:Tensor_Flow
gptkb:Py_Torch
|
|
gptkbp:is_less_likely_to
|
saturate than sigmoid
|
|
gptkbp:is_less_sensitive_to
|
outliers than other functions
|
|
gptkbp:is_often_used_in
|
deep learning models
|
|
gptkbp:is_popular_for
|
hidden layers
|
|
gptkbp:is_recommended_by
|
standard Re LU in some cases
|
|
gptkbp:is_similar_to
|
Parametric Re LU
|
|
gptkbp:is_used_in
|
gptkb:neural_networks
gptkb:generative_adversarial_networks
reinforcement learning
transfer learning
convolutional neural networks
image classification tasks
autoencoders
natural language processing tasks
|
|
gptkbp:modifications
|
linear activation function
|
|
gptkbp:type_of
|
Re LU
|
|
gptkbp:variant
|
Rectified Linear Unit
|
|
gptkbp:bfsParent
|
gptkb:Tiny_YOLOv3
|
|
gptkbp:bfsLayer
|
7
|