Statements (26)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
activation function
|
| gptkbp:approximation |
0.5x(1 + tanh(√(2/π)(x + 0.044715x³)))
|
| gptkbp:contrastsWith |
gptkb:Swish
gptkb:ELU gptkb:ReLU |
| gptkbp:differentiable |
yes
|
| gptkbp:domain |
gptkb:artificial_intelligence
gptkb:machine_learning |
| gptkbp:form |
x * Φ(x)
|
| gptkbp:fullName |
Gaussian Error Linear Unit
|
| https://www.w3.org/2000/01/rdf-schema#label |
GELU
|
| gptkbp:introduced |
gptkb:Kevin_Gimpel
Dan Hendrycks |
| gptkbp:introducedIn |
2016
|
| gptkbp:nonlinear |
yes
|
| gptkbp:notablePublication |
Gaussian Error Linear Units (GELUs)
https://arxiv.org/abs/1606.08415 |
| gptkbp:usedIn |
gptkb:GPT-2
gptkb:BERT deep learning transformer models |
| gptkbp:bfsParent |
gptkb:ViT-B
gptkb:ViT-H gptkb:ViT-L gptkb:ViT-S |
| gptkbp:bfsLayer |
7
|