|
gptkbp:instanceOf
|
Vision Transformer model
|
|
gptkbp:architecture
|
gptkb:transformation
|
|
gptkbp:author
|
gptkb:Armand_Joulin
gptkb:Edouard_Grave
gptkb:Hugo_Touvron
gptkb:Hervé_Jégou
gptkb:Gabriel_Synnaeve
gptkb:Matthieu_Cord
Alexandre Sablayrolles
|
|
gptkbp:citation
|
Over 2000 (as of 2024)
|
|
gptkbp:competitor
|
gptkb:ResNet
gptkb:ViT
|
|
gptkbp:developedBy
|
gptkb:Facebook_AI_Research
|
|
gptkbp:distillationToken
|
Yes
|
|
gptkbp:fullName
|
gptkb:Data-efficient_Image_Transformer
|
|
gptkbp:hasVariant
|
DeiT-Base
DeiT-Large
DeiT-Small
DeiT-Tiny
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
DeiT
|
|
gptkbp:input
|
gptkb:illustrator
|
|
gptkbp:introducedIn
|
2020
|
|
gptkbp:license
|
Apache 2.0
|
|
gptkbp:mainActivity
|
Image classification
|
|
gptkbp:notableFor
|
Data efficiency
Efficient training on smaller datasets
Knowledge distillation with a distillation token
No need for large-scale pretraining
|
|
gptkbp:notablePublication
|
Training data-efficient image transformers & distillation through attention
https://arxiv.org/abs/2012.12877
|
|
gptkbp:openSource
|
Yes
|
|
gptkbp:output
|
Class label
|
|
gptkbp:parameter
|
22 million (DeiT-B)
5.7 million (DeiT-T)
86 million (DeiT-L)
|
|
gptkbp:platform
|
gptkb:PyTorch
|
|
gptkbp:pretrainedWeightsAvailable
|
Yes
|
|
gptkbp:publishedIn
|
gptkb:ICML_2021
|
|
gptkbp:relatedTo
|
gptkb:ImageNet
gptkb:Vision_Transformer
Knowledge distillation
|
|
gptkbp:repository
|
https://github.com/facebookresearch/deit
|
|
gptkbp:resolution
|
224x224 pixels
|
|
gptkbp:trainer
|
gptkb:ImageNet
|
|
gptkbp:usedIn
|
Computer vision research
Fine-tuning for downstream tasks
Transfer learning
|
|
gptkbp:usesDistillation
|
Yes
|
|
gptkbp:bfsParent
|
gptkb:Vision_Transformer
|
|
gptkbp:bfsLayer
|
6
|