Statements (32)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:academic_journal
|
| gptkbp:allows |
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.
|
| gptkbp:author |
gptkb:Jacob_Devlin
gptkb:Kenton_Lee gptkb:Ming-Wei_Chang gptkb:Kristina_Toutanova |
| gptkbp:category |
cs.CL
stat.ML |
| gptkbp:citation |
thousands of scientific papers
|
| gptkbp:contribution |
Demonstrated state-of-the-art results on multiple NLP tasks
Introduced BERT model for NLP Popularized transformer-based pre-training for language understanding |
| https://www.w3.org/2000/01/rdf-schema#label |
arXiv:1810.09434
|
| gptkbp:influencedBy |
gptkb:machine_learning
deep learning natural language processing question answering sentiment analysis text classification named entity recognition language inference transformer models |
| gptkbp:language |
English
|
| gptkbp:license |
arXiv.org perpetual, non-exclusive license
|
| gptkbp:memberSchool |
gptkb:Google_AI_Language
|
| gptkbp:openAccess |
true
|
| gptkbp:pdf |
https://arxiv.org/pdf/1810.09434.pdf
|
| gptkbp:publicationDate |
2018-10-11
|
| gptkbp:title |
gptkb:BERT:_Pre-training_of_Deep_Bidirectional_Transformers_for_Language_Understanding
|
| gptkbp:url |
https://arxiv.org/abs/1810.09434
|
| gptkbp:bfsParent |
gptkb:Unitary_Coupled_Cluster_Method
|
| gptkbp:bfsLayer |
7
|