ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
GPTKB entity
Statements (25)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:academic_journal
large language model |
| gptkbp:affiliation |
gptkb:Stanford_University
gptkb:Google_Research |
| gptkbp:approach |
discriminative pre-training
|
| gptkbp:arXivID |
2003.10555
|
| gptkbp:author |
gptkb:Kevin_Clark
gptkb:Christopher_D._Manning gptkb:Minh-Thang_Luong gptkb:Quoc_V._Le |
| gptkbp:citation |
many subsequent NLP papers
|
| gptkbp:comparesWith |
gptkb:BERT
gptkb:GPT |
| gptkbp:focusesOn |
pre-training text encoders
|
| gptkbp:hasMethod |
gptkb:ELECTRA
|
| https://www.w3.org/2000/01/rdf-schema#label |
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
|
| gptkbp:language |
English
|
| gptkbp:openAccess |
true
|
| gptkbp:proposesAlternativeTo |
masked language modeling
|
| gptkbp:publicationYear |
2020
|
| gptkbp:publishedIn |
gptkb:International_Conference_on_Learning_Representations
|
| gptkbp:technique |
replaced token detection
|
| gptkbp:url |
https://arxiv.org/abs/2003.10555
|
| gptkbp:bfsParent |
gptkb:ELECTRA
|
| gptkbp:bfsLayer |
7
|