GPT-2 (2019)

GPTKB entity

Statements (43)
Predicate Object
gptkbp:instanceOf large language model
gptkbp:architecture gptkb:transformation
gptkbp:author gptkb:Ilya_Sutskever
gptkb:Alec_Radford
gptkb:Dario_Amodei
gptkb:David_Luan
gptkb:Rewon_Child
gptkb:Jeffrey_Wu
gptkbp:contextWindowSize 1024 tokens
gptkbp:developedBy gptkb:OpenAI
https://www.w3.org/2000/01/rdf-schema#label GPT-2 (2019)
gptkbp:input gptkb:text
gptkbp:language English
gptkbp:license gptkb:OpenAI_License
gptkbp:memiliki_tugas text generation
language modeling
gptkbp:notableFor text completion
few-shot learning
multi-task learning
zero-shot learning
controversy over release
large-scale unsupervised learning
coherent long-form text generation
delayed full model release due to misuse concerns
demonstrated risks of large language models
influenced subsequent language models
no fine-tuning required for many tasks
gptkbp:notablePublication gptkb:Language_Models_are_Unsupervised_Multitask_Learners
https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
gptkbp:openSource partially
gptkbp:output gptkb:text
gptkbp:parameter 1.5 billion
gptkbp:predecessor GPT (2018)
gptkbp:releaseYear 2019
gptkbp:size 1.5B
117M
345M
762M
gptkbp:successor gptkb:GPT-3_(2020)
gptkbp:trainer WebText dataset
gptkbp:trainingObjective next word prediction
gptkbp:bfsParent gptkb:OpenAI_models
gptkbp:bfsLayer 7