|
gptkbp:instanceOf
|
gptkb:model
large language model
Natural Language Processing Task
|
|
gptkbp:abbreviation
|
gptkb:NLG
|
|
gptkbp:alternativeName
|
BERT_model
Natural_Language_Generation
language_model
natural_language_processing_model
|
|
gptkbp:application
|
gptkb:Dialogue_Systems
gptkb:Machine_Translation
gptkb:Question_Answering
gptkb:Text_Summarization
gptkb:Data-to-Text_Generation
Chatbots
Report Generation
Story Generation
|
|
gptkbp:architecture
|
gptkb:transformation
|
|
gptkbp:attentionMechanism
|
self-attention
|
|
gptkbp:author
|
gptkb:Jacob_Devlin
gptkb:Kenton_Lee
gptkb:Ming-Wei_Chang
gptkb:Kristina_Toutanova
|
|
gptkbp:bench
|
gptkb:CoLA
gptkb:GLUE
gptkb:MNLI
gptkb:MRPC
gptkb:QNLI
gptkb:SQuAD
gptkb:STS-B
gptkb:SWAG
gptkb:QQP
|
|
gptkbp:bidirectional
|
true
|
|
gptkbp:challenge
|
gptkb:Coherence
Creativity
Fluency
Relevance
Factual Accuracy
|
|
gptkbp:component
|
gptkb:Dialogue_Systems
gptkb:Text-to-Speech_Systems
Conversational AI
|
|
gptkbp:conference
|
gptkb:EMNLP
gptkb:INLG
gptkb:AAAI
gptkb:COLING
gptkb:NAACL
gptkb:EACL
gptkb:ACL
gptkb:IJCAI
|
|
gptkbp:developedBy
|
gptkb:Google_AI
|
|
gptkbp:field
|
gptkb:artificial_intelligence
gptkb:Computational_Linguistics
|
|
gptkbp:fineTuning
|
task-specific datasets
|
|
gptkbp:firstResearch
|
1970s
|
|
gptkbp:fullName
|
gptkb:Bidirectional_Encoder_Representations_from_Transformers
|
|
gptkbp:goal
|
Automate Text Creation
Produce Human-like Text
|
|
gptkbp:influenced
|
gptkb:BioBERT
gptkb:SciBERT
gptkb:ERNIE
gptkb:ALBERT
gptkb:DistilBERT
gptkb:RoBERTa
gptkb:XLNet
|
|
gptkbp:input
|
WordPiece embeddings
sentence pairs
single sentences
|
|
gptkbp:inputLengthLimit
|
512 tokens
|
|
gptkbp:inspiredBy
|
Transformer architecture
|
|
gptkbp:introducedIn
|
2018
|
|
gptkbp:language
|
English
|
|
gptkbp:notableFor
|
gptkb:BART
gptkb:Jasper
gptkb:T5
gptkb:ELIZA
gptkb:ChatGPT
gptkb:GPT-3
gptkb:Narrative_Science_Quill
gptkb:LLaMA
gptkb:BLOOM
gptkb:OpenAI_Codex
gptkb:XLNet
BERT (for some tasks)
|
|
gptkbp:notablePublication
|
gptkb:BERT:_Pre-training_of_Deep_Bidirectional_Transformers_for_Language_Understanding
|
|
gptkbp:openSource
|
true
|
|
gptkbp:organization
|
gptkb:Google
gptkb:IBM
gptkb:Microsoft
gptkb:OpenAI
gptkb:Hugging_Face
gptkb:Cohere
gptkb:Meta
gptkb:Anthropic
gptkb:Narrative_Science
gptkb:AI21_Labs
|
|
gptkbp:output
|
contextualized word embeddings
|
|
gptkbp:parameter
|
110 million (BERT Base)
340 million (BERT Large)
|
|
gptkbp:pretrainingTask
|
masked language modeling
next sentence prediction
|
|
gptkbp:publishedIn
|
gptkb:NAACL_2019
gptkb:Journal_of_Artificial_Intelligence_Research
gptkb:Transactions_of_the_Association_for_Computational_Linguistics
gptkb:Computational_Linguistics
gptkb:Journal_of_Natural_Language_Engineering
gptkb:Natural_Language_Engineering
gptkb:AI_Magazine
|
|
gptkbp:relatedStandard
|
gptkb:SimpleNLG
gptkb:WebNLG
|
|
gptkbp:relatedTo
|
gptkb:Natural_Language_Processing
Natural Language Understanding
|
|
gptkbp:repository
|
https://github.com/google-research/bert
|
|
gptkbp:step
|
Content Determination
Lexicalization
Linguistic Realization
Referring Expression Generation
Sentence Aggregation
Text Structuring
|
|
gptkbp:tokenizer
|
gptkb:WordPiece
|
|
gptkbp:trainer
|
gptkb:English_Wikipedia
gptkb:BooksCorpus
|
|
gptkbp:usedFor
|
question answering
sentiment analysis
sentence similarity
text classification
named entity recognition
language inference
|
|
gptkbp:uses
|
gptkb:Neural_Networks
gptkb:Large_Language_Models
Deep Learning
Templates
Statistical Methods
Rule-based Systems
|
|
gptkbp:bfsParent
|
gptkb:artificial_intelligence
|
|
gptkbp:bfsLayer
|
4
|