Statements (66)
Predicate | Object |
---|---|
gptkbp:instance_of |
gptkb:Transformers_character
|
gptkbp:bfsLayer |
4
|
gptkbp:bfsParent |
gptkb:Transformers_character
|
gptkbp:application |
Medical imaging
Autonomous driving Facial recognition Video analysis Image generation Image classification Object detection Semantic segmentation |
gptkbp:architectural_style |
gptkb:Transformers_character
|
gptkbp:cache_size |
gptkb:32
|
gptkbp:developed_by |
gptkb:Google_Research
|
gptkbp:established |
Ge LU
|
gptkbp:features |
Data augmentation
Fine-tuning self-attention mechanism positional encoding Global average pooling Gradient clipping Attention maps Class token Image normalization Interpretability tools Layer-wise learning rate decay Model ensembling Patch size 16x16 Transferable features patch embeddings |
gptkbp:first_introduced |
Xavier initialization
|
gptkbp:game_components |
Layer normalization
Multi-head attention Residual connections Feed-forward network |
gptkbp:has_website |
Computer vision
|
https://www.w3.org/2000/01/rdf-schema#label |
Vi T-B
|
gptkbp:influenced_by |
gptkb:Attention_is_All_You_Need
|
gptkbp:input_output |
224x224
|
gptkbp:is_evaluated_by |
gptkb:CIFAR-10
gptkb:Stanford_Dogs gptkb:CIFAR-100 F1 score Oxford Pets |
gptkbp:losses |
Cross-entropy loss
|
gptkbp:orbital_period |
86 million
|
gptkbp:performance |
Top-1 accuracy
88.55% |
gptkbp:predecessor |
CN Ns
|
gptkbp:provides_information_on |
gptkb:Image_Net
1.2 million images |
gptkbp:reduces |
0.1
|
gptkbp:related_to |
Deep learning
Neural networks |
gptkbp:release_year |
gptkb:2020
|
gptkbp:resolution |
1000 classes
224x224 pixels |
gptkbp:successor |
gptkb:Vi_T-L
|
gptkbp:training |
gptkb:Graphics_Processing_Unit
gptkb:Py_Torch 0.001 Supervised learning |
gptkbp:tuning |
gptkb:Adam_optimizer
|
gptkbp:uses |
Transfer learning
Self-Attention Mechanism |
gptkbp:values |
86 million
|