Multilingual BERT

GPTKB entity

Statements (46)
Predicate Object
gptkbp:instanceOf gptkb:model
large language model
gptkbp:architecture gptkb:transformation
gptkbp:attentionHeads 12
gptkbp:availableOn gptkb:Hugging_Face
gptkb:TensorFlow_Hub
gptkbp:basedOn gptkb:BERT
gptkbp:category gptkb:model
gptkb:NLP
deep learning
multilingual model
pretrained model
gptkbp:citation gptkb:Devlin_et_al.,_2018
gptkbp:developedBy gptkb:Google
gptkbp:embeddingSize 768
gptkbp:hasVariant cased
uncased
https://www.w3.org/2000/01/rdf-schema#label Multilingual BERT
gptkbp:input gptkb:text
gptkbp:inputLengthLimit 512 tokens
gptkbp:layer 12
gptkbp:license Apache 2.0
gptkbp:notTrainedOn parallel corpora
gptkbp:openSource true
gptkbp:output contextual embeddings
gptkbp:parameter 110 million
340 million
gptkbp:predecessor gptkb:BERT
gptkbp:releaseYear 2018
gptkbp:successor gptkb:mBERT_(large)
gptkb:XLM-R
gptkbp:supportsLanguage 104 languages
gptkbp:tokenizerType gptkb:WordPiece
gptkbp:trainer gptkb:Wikipedia
masked language modeling
next sentence prediction
Wikipedia (104 languages)
gptkbp:usedFor natural language processing
question answering
text classification
named entity recognition
gptkbp:usedIn zero-shot learning
cross-lingual transfer learning
gptkbp:website https://github.com/google-research/bert
gptkbp:bfsParent gptkb:BERT
gptkbp:bfsLayer 6