transformer (machine learning model)

URI: https://gptkb.org/entity/transformer_(machine_learning_model)

GPTKB entity

Statements (100)

Predicate	Object
gptkbp:instanceOf	gptkb:convolutional_neural_network
gptkbp:architecture	encoder-decoder
gptkbp:component	feed-forward neural network layer normalization multi-head attention positional encoding residual connection self-attention mechanism
gptkbp:decoderComponent	decoder
gptkbp:encoderComponent	encoder
gptkbp:features	attention mechanism scalability parallelization long-range dependency modeling
https://www.w3.org/2000/01/rdf-schema#label	transformer (machine learning model)
gptkbp:input	integer sequence
gptkbp:inspiredBy	gptkb:T5 gptkb:BERT gptkb:GPT gptkb:RoBERTa gptkb:Vision_Transformer gptkb:XLNet
gptkbp:introduced	gptkb:Attention_Is_All_You_Need gptkb:Vaswani_et_al.
gptkbp:introducedIn	2017
gptkbp:notableFor	gptkb:BART gptkb:Google_Translate gptkb:Chinchilla gptkb:Pegasus gptkb:T5 gptkb:Yi gptkb:Gemini_Nano gptkb:Gemini_Pro gptkb:Gemini_Ultra gptkb:Gemini gptkb:Switch_Transformer gptkb:XLM gptkb:XLM-R gptkb:UL2 gptkb:Bing_AI gptkb:DBRX gptkb:Flan-T5 gptkb:Gemini_1.0 gptkb:Gemini_1.5 gptkb:Groq_LPU gptkb:MarianMT gptkb:Phi-2 gptkb:mBART gptkb:AlphaFold gptkb:ChatGPT gptkb:DALL-E gptkb:GPT-2 gptkb:GPT-3 gptkb:GPT-4 gptkb:Sparrow gptkb:Gopher gptkb:ELECTRA gptkb:Claude gptkb:LLaMA gptkb:Mistral gptkb:Stable_Diffusion gptkb:ERNIE gptkb:LaMDA gptkb:PaLM gptkb:Gemma gptkb:BERT gptkb:BLOOM gptkb:Claude_3 gptkb:GPT-4o gptkb:OPT gptkb:ALBERT gptkb:BigBird gptkb:DistilBERT gptkb:GPT-J gptkb:GPT-Neo gptkb:Longformer gptkb:Mixtral gptkb:Qwen gptkb:RoBERTa gptkb:XLNet actor Reformer
gptkbp:openSource	gptkb:TensorFlow gptkb:PyTorch gptkb:Hugging_Face_Transformers
gptkbp:output	integer sequence
gptkbp:replacedBy	gptkb:long_short-term_memory recurrent neural network gated recurrent unit
gptkbp:trainer	supervised learning unsupervised learning large datasets
gptkbp:type	scaled dot-product attention
gptkbp:usedFor	machine translation natural language processing speech recognition image processing text generation
gptkbp:bfsParent	gptkb:convolutional_neural_network
gptkbp:bfsLayer	5