Google Switch Transformer

URI: https://gptkb.org/entity/Google_Switch_Transformer

GPTKB entity

Predicate	Object
gptkbp:instanceOf	gptkb:convolutional_neural_network
gptkbp:announced	2021
gptkbp:author	gptkb:Noam_Shazeer gptkb:Barret_Zoph gptkb:William_Fedus
gptkbp:citation	1000+
gptkbp:developedBy	gptkb:Google
gptkbp:enables	scaling to trillion-parameter models
gptkbp:feature	activates only a subset of model parameters per input
gptkbp:improves	gptkb:T5 gptkb:GPT-3 dense transformer models
gptkbp:notablePublication	gptkb:Switch_Transformers:_Scaling_to_Trillion_Parameter_Models_with_Simple_and_Efficient_Sparsity
gptkbp:openSource	no
gptkbp:organization	gptkb:Google_Research
gptkbp:parameter	1.6 trillion
gptkbp:publishedIn	gptkb:arXiv
gptkbp:purpose	natural language processing language modeling
gptkbp:reduces	computational cost
gptkbp:relatedTo	gptkb:T5 gptkb:GPT-3 gptkb:Mixture_of_Experts
gptkbp:type	gptkb:model sparse model
gptkbp:uses	gptkb:Mixture_of_Experts
gptkbp:bfsParent	gptkb:Mixture_of_Experts gptkb:Mixture_of_experts
gptkbp:bfsLayer	8
https://www.w3.org/2000/01/rdf-schema#label	Google Switch Transformer