|
gptkbp:instanceOf
|
gptkb:model
large language model
|
|
gptkbp:architecture
|
gptkb:transformation
|
|
gptkbp:author
|
gptkb:Mike_Lewis
gptkb:Omer_Levy
gptkb:Danqi_Chen
gptkb:Jingfei_Du
gptkb:Mandar_Joshi
gptkb:Myle_Ott
gptkb:Yinhan_Liu
gptkb:Veselin_Stoyanov
gptkb:Luke_Zettlemoyer
gptkb:Naman_Goyal
|
|
gptkbp:availableOn
|
gptkb:Hugging_Face
|
|
gptkbp:basedOn
|
gptkb:BERT
|
|
gptkbp:category
|
gptkb:machine_learning
deep learning
natural language processing
|
|
gptkbp:compatibleWith
|
next sentence prediction
|
|
gptkbp:developedBy
|
gptkb:Facebook_AI
|
|
gptkbp:fineTunedWith
|
question answering
sentiment analysis
text classification
named entity recognition
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
RoBERTa
|
|
gptkbp:improves
|
gptkb:BERT
|
|
gptkbp:input
|
gptkb:text
|
|
gptkbp:language
|
English
|
|
gptkbp:license
|
gptkb:MIT_License
|
|
gptkbp:notablePublication
|
gptkb:RoBERTa:_A_Robustly_Optimized_BERT_Pretraining_Approach
|
|
gptkbp:openSource
|
true
|
|
gptkbp:parameter
|
1.5 billion
125 million
2.9 billion
355 million
|
|
gptkbp:pretrainingObjective
|
masked language modeling
|
|
gptkbp:releaseYear
|
2019
|
|
gptkbp:trainer
|
gptkb:English_Wikipedia
gptkb:OpenWebText
gptkb:BookCorpus
gptkb:CC-News
Stories
dynamic masking
larger mini-batches
longer sequences
more data
|
|
gptkbp:uses
|
self-attention
|
|
gptkbp:bfsParent
|
gptkb:large_language_model
gptkb:transformation
gptkb:convolutional_neural_network
gptkb:Large_Language_Models
|
|
gptkbp:bfsLayer
|
5
|