Statements (16)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
activation function
|
| gptkbp:abbreviation |
gptkb:SwiGLU
|
| gptkbp:category |
neural network component
|
| gptkbp:form |
SwiGLU(x) = (xW1) * sigmoid(xW2)
|
| https://www.w3.org/2000/01/rdf-schema#label |
Switch Gated Linear Unit
|
| gptkbp:improves |
model performance
|
| gptkbp:proposedBy |
2020
Shazeer, Noam |
| gptkbp:publishedIn |
arXiv:2002.05202
|
| gptkbp:relatedTo |
gptkb:GELU
Gated Linear Unit |
| gptkbp:usedIn |
deep learning
language models transformer models |
| gptkbp:bfsParent |
gptkb:SwiGLU
|
| gptkbp:bfsLayer |
7
|