Statements (25)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
activation function
|
| gptkbp:alternativeFormula |
x * Φ(x), where Φ(x) is the standard normal CDF
|
| gptkbp:contrastsWith |
gptkb:Swish
gptkb:ELU gptkb:ReLU |
| gptkbp:differentiable |
yes
|
| gptkbp:domain |
gptkb:artificial_intelligence
gptkb:machine_learning |
| gptkbp:form |
x * 0.5 * (1 + tanh(√(2/π) * (x + 0.044715x^3)))
|
| gptkbp:fullName |
Gaussian Error Linear Unit
|
| https://www.w3.org/2000/01/rdf-schema#label |
Gelu
|
| gptkbp:introducedIn |
2016
|
| gptkbp:nonlinear |
yes
|
| gptkbp:notablePublication |
Gaussian Error Linear Units (GELUs)
https://arxiv.org/abs/1606.08415 |
| gptkbp:proposedBy |
gptkb:Kevin_Gimpel
Dan Hendrycks |
| gptkbp:usedIn |
gptkb:GPT-2
gptkb:BERT deep learning transformer models |
| gptkbp:bfsParent |
gptkb:Glatorian_Legends
gptkb:Lego_Bionicle gptkb:Bionicle_2009_storyline |
| gptkbp:bfsLayer |
7
|