TinyBERT: Distilling BERT for Natural Language Understanding
GPTKB entity
Statements (28)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:academic_journal
|
| gptkbp:application |
mobile and edge devices
|
| gptkbp:arXivID |
1909.10351
|
| gptkbp:author |
Chen, Xiao
Jiang, Xin Jiao, Xiaoqi Li, Linlin Shang, Lifeng Wang, Fang Wang, Qun Yin, Yichun |
| gptkbp:focusesOn |
natural language understanding
knowledge distillation model compression |
| https://www.w3.org/2000/01/rdf-schema#label |
TinyBERT: Distilling BERT for Natural Language Understanding
|
| gptkbp:improves |
gptkb:BERT
|
| gptkbp:proposedBy |
gptkb:TinyBERT
embedding distillation layer-to-layer distillation prediction distillation transformer distillation two-stage learning framework |
| gptkbp:publishedIn |
gptkb:EMNLP_2020
|
| gptkbp:title |
gptkb:TinyBERT:_Distilling_BERT_for_Natural_Language_Understanding
|
| gptkbp:url |
https://arxiv.org/abs/1909.10351
|
| gptkbp:year |
2020
|
| gptkbp:bfsParent |
gptkb:TinyBERT
|
| gptkbp:bfsLayer |
6
|