Statements (30)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:large_language_model
gptkb:model |
| gptkbp:accuracyComparedToBERT |
97% of BERT
|
| gptkbp:architecture |
gptkb:transformation
|
| gptkbp:attentionHeads |
12
|
| gptkbp:availableOn |
gptkb:Hugging_Face_Model_Hub
|
| gptkbp:basedOn |
gptkb:BERT
|
| gptkbp:compressionRatio |
knowledge distillation
|
| gptkbp:developedBy |
gptkb:Hugging_Face
|
| gptkbp:hiddenSize |
768
|
| gptkbp:language |
English
|
| gptkbp:layer |
6
|
| gptkbp:license |
Apache 2.0
|
| gptkbp:memiliki_tugas |
feature extraction
question answering text classification named entity recognition |
| gptkbp:openSource |
true
|
| gptkbp:parameter |
66 million
|
| gptkbp:releaseYear |
2019
|
| gptkbp:sequence |
512
|
| gptkbp:sizeComparedToBERT |
40% smaller
|
| gptkbp:speedComparedToBERT |
60% faster
|
| gptkbp:trainer |
gptkb:Wikipedia
gptkb:BookCorpus |
| gptkbp:bfsParent |
gptkb:Hugging_Face
gptkb:Stanford_Question_Answering_Dataset_(SQuAD) gptkb:Cloudflare_Workers_AI |
| gptkbp:bfsLayer |
6
|
| https://www.w3.org/2000/01/rdf-schema#label |
DistilBERT
|