|
gptkbp:instanceOf
|
gptkb:model
large language model
|
|
gptkbp:application
|
question answering
sentiment analysis
text classification
language understanding
natural language inference
|
|
gptkbp:architecture
|
gptkb:transformation
|
|
gptkbp:author
|
gptkb:Jaime_Carbonell
gptkb:Ruslan_Salakhutdinov
gptkb:Yiming_Yang
gptkb:Samy_Bengio
gptkb:Zhilin_Yang
gptkb:Zihang_Dai
gptkb:Quoc_V._Le
|
|
gptkbp:basedOn
|
Transformer architecture
|
|
gptkbp:developedBy
|
gptkb:Carnegie_Mellon_University
gptkb:Google_Brain
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
XLNet
|
|
gptkbp:improves
|
gptkb:GPT-2
gptkb:ERNIE
gptkb:BERT
gptkb:RoBERTa
|
|
gptkbp:input
|
gptkb:text
|
|
gptkbp:language
|
English
|
|
gptkbp:license
|
Apache 2.0
|
|
gptkbp:notableFor
|
permutation-based training objective
state-of-the-art results in 2019
|
|
gptkbp:notablePublication
|
gptkb:XLNet:_Generalized_Autoregressive_Pretraining_for_Language_Understanding
https://arxiv.org/abs/1906.08237
|
|
gptkbp:openSource
|
true
|
|
gptkbp:output
|
predicted tokens
text embeddings
|
|
gptkbp:parameter
|
110 million (XLNet-Base)
340 million (XLNet-Large)
|
|
gptkbp:predecessor
|
gptkb:BERT
|
|
gptkbp:publicationYear
|
2019
|
|
gptkbp:repository
|
https://github.com/zihangdai/xlnet
|
|
gptkbp:supports
|
transfer learning
fine-tuning
|
|
gptkbp:trainer
|
gptkb:Wikipedia
gptkb:ClueWeb_09
gptkb:Giga5
gptkb:Common_Crawl
gptkb:BooksCorpus
|
|
gptkbp:uses
|
autoregressive modeling
permutation language modeling
relative positional encoding
segment recurrence mechanism
|
|
gptkbp:bfsParent
|
gptkb:large_language_model
gptkb:transformation
gptkb:convolutional_neural_network
gptkb:Large_Language_Models
|
|
gptkbp:bfsLayer
|
5
|