Statements (16)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
activation function
|
| gptkbp:combines |
gptkb:GELU
gptkb:GLU |
| gptkbp:component |
feedforward neural networks
|
| gptkbp:fullName |
Gated Linear Unit with GELU
|
| https://www.w3.org/2000/01/rdf-schema#label |
GeGLU
|
| gptkbp:input |
two linear projections
|
| gptkbp:introducedIn |
gptkb:Shazeer_2020
|
| gptkbp:output |
elementwise product of linear and GELU-projected input
|
| gptkbp:purpose |
improve model performance
|
| gptkbp:usedBy |
gptkb:PaLM
gptkb:GPT-NeoX T5.1.1 |
| gptkbp:usedIn |
transformer models
|
| gptkbp:bfsParent |
gptkb:SwiGLU
|
| gptkbp:bfsLayer |
7
|