|
gptkbp:instanceOf
|
Transformer model
|
|
gptkbp:application
|
translation
question answering
text summarization
text classification
|
|
gptkbp:architect
|
Text-to-Text Transfer Transformer
|
|
gptkbp:code
|
available on GitHub
|
|
gptkbp:communitySupport
|
active
|
|
gptkbp:competitors
|
gptkb:T5-Base
gptkb:T5-Large
gptkb:T5-11B
T5-Small
|
|
gptkbp:dataUsage
|
domain-specific datasets
|
|
gptkbp:developedBy
|
gptkb:Google_Research
|
|
gptkbp:evaluates
|
F1 score
ROUGE score
BLEU_score
|
|
gptkbp:gameLength
|
512 tokens
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
T5-3B
|
|
gptkbp:improves
|
possible
|
|
gptkbp:inputOutput
|
softmax
|
|
gptkbp:keyIssues
|
3 billion
|
|
gptkbp:language
|
English
pretrained
|
|
gptkbp:learnsMove
|
variable
|
|
gptkbp:litterSize
|
32
|
|
gptkbp:losses
|
cross-entropy loss
|
|
gptkbp:maximumDepth
|
512 tokens
|
|
gptkbp:model
|
encoder-decoder
|
|
gptkbp:outflow
|
0.1
|
|
gptkbp:powerOutput
|
text
|
|
gptkbp:processor
|
TPUs
|
|
gptkbp:project
|
natural language processing
|
|
gptkbp:propagationMethod
|
gptkb:SentencePiece
|
|
gptkbp:providesTrainingFor
|
C4 dataset
|
|
gptkbp:relatedPatent
|
dialog systems
sentiment analysis
chatbots
data extraction
content generation
|
|
gptkbp:relatedTo
|
BERT
GPT-3
RoBERTa
XLNet
|
|
gptkbp:releaseDate
|
2020
|
|
gptkbp:researchInterest
|
high
|
|
gptkbp:training
|
supervised learning
weeks
|
|
gptkbp:trainingPrograms
|
750 GB
|
|
gptkbp:tuning
|
gptkb:Adam
|
|
gptkbp:uses
|
transfer learning
|