|
gptkbp:instanceOf
|
gptkb:convolutional_neural_network
|
|
gptkbp:application
|
text generation
language modeling
long-context sequence modeling
|
|
gptkbp:author
|
gptkb:Ruslan_Salakhutdinov
gptkb:Yiming_Yang
gptkb:William_W._Cohen
gptkb:Jamie_Carbonell
gptkb:Zhilin_Yang
gptkb:Zihang_Dai
gptkb:Quoc_V._Le
|
|
gptkbp:basedOn
|
Transformer architecture
|
|
gptkbp:citation
|
over 3000 (as of 2024)
|
|
gptkbp:developedBy
|
gptkb:Google_Brain
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
Transformer-XL
|
|
gptkbp:improves
|
context length modeling
previous Transformer models on long-context tasks
|
|
gptkbp:introduced
|
recurrence mechanism
|
|
gptkbp:introducedIn
|
2019
|
|
gptkbp:language
|
gptkb:Python
|
|
gptkbp:license
|
Apache 2.0
|
|
gptkbp:notableFeature
|
relative positional encoding
segment-level recurrence
|
|
gptkbp:notablePublication
|
gptkb:Transformer-XL:_Attentive_Language_Models_Beyond_a_Fixed-Length_Context
|
|
gptkbp:openSource
|
yes
|
|
gptkbp:repository
|
https://github.com/kimiyoung/transformer-xl
|
|
gptkbp:usedIn
|
gptkb:reinforcement_learning
natural language processing
music modeling
|
|
gptkbp:bfsParent
|
gptkb:transformation
gptkb:convolutional_neural_network
|
|
gptkbp:bfsLayer
|
5
|