Statements (28)
Predicate | Object |
---|---|
gptkbp:instanceOf |
large language model
|
gptkbp:architecture |
encoder
|
gptkbp:author |
gptkb:Kevin_Clark
gptkb:Christopher_D._Manning gptkb:Minh-Thang_Luong gptkb:Quoc_V._Le |
gptkbp:availableOn |
gptkb:Hugging_Face_Model_Hub
|
gptkbp:basedOn |
transformer architecture
|
gptkbp:contrastsWith |
gptkb:BERT
gptkb:RoBERTa |
gptkbp:developedBy |
gptkb:Google_Research
|
gptkbp:github |
https://github.com/google-research/electra
|
https://www.w3.org/2000/01/rdf-schema#label |
ELECTRA-Large
|
gptkbp:improves |
BERT-Large (on some benchmarks)
|
gptkbp:introducedIn |
2020
|
gptkbp:language |
English
|
gptkbp:notablePublication |
gptkb:ELECTRA:_Pre-training_Text_Encoders_as_Discriminators_Rather_Than_Generators
|
gptkbp:parameter |
335 million
|
gptkbp:pretrainingMethod |
replaced token detection
|
gptkbp:relatedTo |
gptkb:ELECTRA
|
gptkbp:trainer |
gptkb:Wikipedia
gptkb:BooksCorpus |
gptkbp:usedFor |
question answering
natural language understanding text classification named entity recognition |
gptkbp:bfsParent |
gptkb:ELECTRA
|
gptkbp:bfsLayer |
7
|