|
gptkbp:instanceOf
|
multilingual language model
|
|
gptkbp:application
|
natural language processing
question answering
text classification
named entity recognition
cross-lingual transfer
|
|
gptkbp:architecture
|
gptkb:transformation
|
|
gptkbp:attentionHeads
|
12
|
|
gptkbp:author
|
gptkb:Jacob_Devlin
gptkb:Kenton_Lee
gptkb:Ming-Wei_Chang
gptkb:Kristina_Toutanova
|
|
gptkbp:availableOn
|
gptkb:Hugging_Face
|
|
gptkbp:basedOn
|
gptkb:BERT
|
|
gptkbp:citation
|
gptkb:BERT:_Pre-training_of_Deep_Bidirectional_Transformers_for_Language_Understanding
2019
NAACL-HLT 2019
|
|
gptkbp:developedBy
|
gptkb:Google
|
|
gptkbp:hiddenLayers
|
12
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
mBERT
|
|
gptkbp:input
|
gptkb:text
|
|
gptkbp:license
|
Apache 2.0
|
|
gptkbp:openSource
|
true
|
|
gptkbp:parameter
|
110 million
|
|
gptkbp:pretrainingObjective
|
masked language modeling
next sentence prediction
|
|
gptkbp:relatedTo
|
gptkb:XLM-R
gptkb:BERT
gptkb:DistilBERT
|
|
gptkbp:releaseYear
|
2019
|
|
gptkbp:supportsLanguage
|
104 languages
|
|
gptkbp:tokenizer
|
gptkb:WordPiece
|
|
gptkbp:trainer
|
gptkb:Wikipedia
|
|
gptkbp:bfsParent
|
gptkb:XLM-R
|
|
gptkbp:bfsLayer
|
6
|