Cross-lingual Language Model Pretraining
GPTKB entity
Statements (49)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
machine learning technique
|
| gptkbp:application |
multilingual NLP
|
| gptkbp:approach |
self-supervised learning
supervised learning unsupervised learning |
| gptkbp:architecture |
gptkb:transformation
encoder-decoder decoder-only encoder-only |
| gptkbp:benefit |
language transfer
multilingual applications improved performance on low-resource languages |
| gptkbp:challenge |
data imbalance
low-resource languages alignment of representations tokenization across languages |
| gptkbp:enables |
zero-shot learning
cross-lingual transfer multilingual understanding |
| gptkbp:field |
natural language processing
|
| gptkbp:goal |
learn representations across languages
|
| https://www.w3.org/2000/01/rdf-schema#label |
Cross-lingual Language Model Pretraining
|
| gptkbp:improves |
machine translation
named entity recognition cross-lingual question answering cross-lingual text classification |
| gptkbp:key |
gptkb:Unsupervised_Cross-lingual_Representation_Learning_at_Scale_(Conneau_et_al.,_2020)
gptkb:XLM:_Cross-lingual_Language_Model_Pretraining_(Lample_and_Conneau,_2019) |
| gptkbp:notableModel |
gptkb:mBERT
gptkb:XLM gptkb:XLM-R gptkb:InfoXLM gptkb:LaBSE gptkb:Unicoder |
| gptkbp:proposedBy |
2019
|
| gptkbp:relatedTo |
gptkb:mBERT
gptkb:XLM gptkb:XLM-R gptkb:BERT |
| gptkbp:usedBy |
gptkb:Google
gptkb:Microsoft_Research gptkb:Hugging_Face gptkb:Facebook_AI |
| gptkbp:uses |
masked language modeling
monolingual corpora parallel corpora translation language modeling |
| gptkbp:bfsParent |
gptkb:Guillaume_Lample
|
| gptkbp:bfsLayer |
6
|