|
gptkbp:instanceOf
|
gptkb:large_language_model
|
|
gptkbp:architecture
|
encoder
|
|
gptkbp:author
|
gptkb:Kevin_Clark
gptkb:Christopher_D._Manning
gptkb:Minh-Thang_Luong
gptkb:Quoc_V._Le
|
|
gptkbp:availableOn
|
gptkb:Hugging_Face_Model_Hub
|
|
gptkbp:basedOn
|
transformer architecture
|
|
gptkbp:contrastsWith
|
gptkb:BERT
gptkb:RoBERTa
|
|
gptkbp:developedBy
|
gptkb:Google_Research
|
|
gptkbp:github
|
https://github.com/google-research/electra
|
|
gptkbp:improves
|
BERT-Large (on some benchmarks)
|
|
gptkbp:introducedIn
|
2020
|
|
gptkbp:language
|
English
|
|
gptkbp:notablePublication
|
gptkb:ELECTRA:_Pre-training_Text_Encoders_as_Discriminators_Rather_Than_Generators
|
|
gptkbp:parameter
|
335 million
|
|
gptkbp:pretrainingMethod
|
replaced token detection
|
|
gptkbp:relatedTo
|
gptkb:ELECTRA
|
|
gptkbp:trainer
|
gptkb:Wikipedia
gptkb:BooksCorpus
|
|
gptkbp:usedFor
|
question answering
natural language understanding
text classification
named entity recognition
|
|
gptkbp:bfsParent
|
gptkb:ELECTRA
|
|
gptkbp:bfsLayer
|
8
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
ELECTRA-Large
|