Decoding-enhanced BERT with Disentangled Attention
GPTKB entity
Statements (28)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:academic_journal
|
| gptkbp:abbreviation |
gptkb:DeBERTa
|
| gptkbp:author |
gptkb:Pengcheng_He
gptkb:Weizhu_Chen gptkb:Xiaodong_Liu gptkb:Jianfeng_Gao |
| gptkbp:basedOn |
gptkb:BERT
|
| gptkbp:citation |
1000+
|
| gptkbp:field |
gptkb:Machine_Learning
gptkb:Natural_Language_Processing |
| https://www.w3.org/2000/01/rdf-schema#label |
Decoding-enhanced BERT with Disentangled Attention
|
| gptkbp:improves |
gptkb:BERT
gptkb:RoBERTa gptkb:XLNet |
| gptkbp:introduced |
disentangled attention mechanism
enhanced mask decoder |
| gptkbp:language |
English
|
| gptkbp:proposedBy |
gptkb:Microsoft_Research
|
| gptkbp:publicationYear |
2020
|
| gptkbp:relatedTo |
transformer models
pre-trained language models |
| gptkbp:repository |
https://github.com/microsoft/DeBERTa
|
| gptkbp:usedFor |
gptkb:GLUE_benchmark
gptkb:SQuAD gptkb:SuperGLUE language understanding tasks |
| gptkbp:bfsParent |
gptkb:DeBERTa
|
| gptkbp:bfsLayer |
6
|