Statements (30)
Predicate | Object |
---|---|
gptkbp:instanceOf |
gptkb:model
large language model |
gptkbp:accuracyComparedToBERT |
97% of BERT
|
gptkbp:architecture |
gptkb:transformation
|
gptkbp:attentionHeads |
12
|
gptkbp:availableOn |
gptkb:Hugging_Face_Model_Hub
|
gptkbp:basedOn |
gptkb:BERT
|
gptkbp:compressionRatio |
knowledge distillation
|
gptkbp:developedBy |
gptkb:Hugging_Face
|
gptkbp:hiddenSize |
768
|
https://www.w3.org/2000/01/rdf-schema#label |
DistilBERT
|
gptkbp:language |
English
|
gptkbp:layer |
6
|
gptkbp:license |
Apache 2.0
|
gptkbp:memiliki_tugas |
feature extraction
question answering text classification named entity recognition |
gptkbp:openSource |
true
|
gptkbp:parameter |
66 million
|
gptkbp:releaseYear |
2019
|
gptkbp:sequence |
512
|
gptkbp:sizeComparedToBERT |
40% smaller
|
gptkbp:speedComparedToBERT |
60% faster
|
gptkbp:trainer |
gptkb:Wikipedia
gptkb:BookCorpus |
gptkbp:bfsParent |
gptkb:large_language_model
gptkb:transformation gptkb:convolutional_neural_network |
gptkbp:bfsLayer |
5
|