Attention Is All You Need

URI: https://gptkb.org/entity/Attention_Is_All_You_Need

GPTKB entity

Statements (50)

Predicate	Object
gptkbp:instanceOf	gptkb:academic_journal
gptkbp:affiliation	gptkb:University_of_Toronto gptkb:Google_Brain
gptkbp:arXivID	1706.03762
gptkbp:author	gptkb:Illia_Polosukhin gptkb:Łukasz_Kaiser gptkb:Aidan_N._Gomez gptkb:Ashish_Vaswani gptkb:Jakob_Uszkoreit gptkb:Llion_Jones gptkb:Niki_Parmar gptkb:Noam_Shazeer
gptkbp:citation	gptkb:BERT:_Pre-training_of_Deep_Bidirectional_Transformers_for_Language_Understanding gptkb:T5:_Exploring_the_Limits_of_Transfer_Learning_with_a_Unified_Text-to-Text_Transformer gptkb:GPT:_Improving_Language_Understanding_by_Generative_Pre-Training over 80,000
gptkbp:contribution	self-attention mechanism improved translation quality parallelization of training removal of recurrence in sequence modeling
gptkbp:doi	10.48550/arXiv.1706.03762
gptkbp:field	gptkb:machine_learning natural language processing
gptkbp:focusesOn	neural machine translation
gptkbp:hasSection	Conclusion Introduction Results Training Background References Model Architecture
https://www.w3.org/2000/01/rdf-schema#label	Attention Is All You Need
gptkbp:impact	enabled large language models revolutionized NLP
gptkbp:influenced	gptkb:T5 gptkb:BERT gptkb:GPT gptkb:XLNet
gptkbp:language	English
gptkbp:license	arXiv.org perpetual, non-exclusive license
gptkbp:openAccess	yes
gptkbp:pages	11
gptkbp:proposedBy	Transformer architecture
gptkbp:publicationYear	2017
gptkbp:publishedIn	gptkb:Advances_in_Neural_Information_Processing_Systems_(NeurIPS)
gptkbp:trainer	gptkb:WMT_2014_English-to-French gptkb:WMT_2014_English-to-German
gptkbp:url	https://arxiv.org/abs/1706.03762
gptkbp:bfsParent	gptkb:Large_Language_Models
gptkbp:bfsLayer	5