DeiT

E435881

DeiT is a family of data-efficient vision transformer models designed for image classification with reduced training data requirements and strong performance.

All labels observed (9)

How this entity was disambiguated

Statements (48)

Predicate Object
instanceOf deep learning model
image classification model
vision transformer model family
abbreviationFor Data-efficient Image Transformers
linked to: DeiT
availableIn Hugging Face Transformers
timm library
basedOn ViT architecture
Vision Transformer
linked to: ViT
codeRepository https://github.com/facebookresearch/deit
contribution reduced data requirements for vision transformers
showed transformers can be trained from scratch on ImageNet-1k only
designedFor image classification
developedBy Facebook AI Research
linked to: Meta AI

Meta AI
doesNotRequire ImageNet-21k pretraining
JFT pretraining
firstAuthor Hugo Touvron
fullName Data-efficient Image Transformers
linked to: DeiT
hasAuthor Alexandre Sablayrolles
Gabriel Synnaeve
Herve Jegou
Hugo Touvron
Matthieu Cord
hasProperty competitive accuracy on ImageNet
reduced training data requirements
strong performance with limited data
hasVariant DeiT-B
linked to: DeiT

DeiT-B distilled
linked to: DeiT

DeiT-S
linked to: DeiT

DeiT-S distilled
linked to: DeiT

DeiT-Ti
linked to: DeiT

DeiT-Ti distilled
linked to: DeiT
implementedIn PyTorch
inputType 2D images
introducedDistillationMethod token-based knowledge distillation
license Apache-2.0 (for official code release)
optimizedFor data efficiency
paperTitle Training data-efficient image transformers & distillation through attention
linked to: DeiT
publicationYear 2020
publishedAs research paper
task supervised image classification
teacherModelType convolutional neural network
trainingDataset ImageNet-1k
linked to: ImageNet
uses class token
distillation token
knowledge distillation from CNN teacher
patch embedding
transformer encoder

How these facts were elicited

Referenced by (11)

Full triples — surface form annotated when it differs from this entity's canonical label.

ViT hasVariant DeiT
DeiT fullName Data-efficient Image Transformers
linked to: DeiT
DeiT abbreviationFor Data-efficient Image Transformers
linked to: DeiT
DeiT paperTitle Training data-efficient image transformers & distillation through attention
linked to: DeiT
DeiT hasVariant DeiT-Ti
linked to: DeiT
DeiT hasVariant DeiT-S
linked to: DeiT
DeiT hasVariant DeiT-B
linked to: DeiT
DeiT hasVariant DeiT-Ti distilled
linked to: DeiT
DeiT hasVariant DeiT-S distilled
linked to: DeiT
DeiT hasVariant DeiT-B distilled
linked to: DeiT