|
gptkbp:instanceOf
|
gptkb:large_language_model
|
|
gptkbp:architecture
|
gptkb:transformation
|
|
gptkbp:author
|
gptkb:Kevin_Clark
gptkb:Christopher_D._Manning
gptkb:Minh-Thang_Luong
gptkb:Quoc_V._Le
|
|
gptkbp:availableOn
|
gptkb:GitHub
|
|
gptkbp:citation
|
high
|
|
gptkbp:contrastsWith
|
gptkb:BERT
gptkb:RoBERTa
gptkb:XLNet
|
|
gptkbp:developedBy
|
gptkb:Google_Research
|
|
gptkbp:hasVariant
|
gptkb:ELECTRA-Base
gptkb:ELECTRA-Large
gptkb:ELECTRA-Small
|
|
gptkbp:input
|
gptkb:text
|
|
gptkbp:language
|
English
|
|
gptkbp:notableFor
|
sample efficiency
competitive performance with less compute
faster pretraining
|
|
gptkbp:notablePublication
|
gptkb:ELECTRA:_Pre-training_Text_Encoders_as_Discriminators_Rather_Than_Generators
|
|
gptkbp:openSource
|
yes
|
|
gptkbp:output
|
contextual embeddings
|
|
gptkbp:pretrainingMethod
|
replaced token detection
|
|
gptkbp:releaseYear
|
2020
|
|
gptkbp:trainer
|
gptkb:Wikipedia
gptkb:BooksCorpus
|
|
gptkbp:usedFor
|
question answering
text classification
named entity recognition
language understanding
|
|
gptkbp:bfsParent
|
gptkb:NLP
|
|
gptkbp:bfsLayer
|
7
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
ELECTRA
|