Statements (29)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
large language model framework
|
| gptkbp:compatibleWith |
gptkb:PyTorch
|
| gptkbp:developedBy |
gptkb:NVIDIA
|
| gptkbp:feature |
distributed training
data parallelism mixed precision training model parallelism pipeline parallelism |
| gptkbp:firstReleased |
2019
|
| https://www.w3.org/2000/01/rdf-schema#label |
NVIDIA Megatron-LM
|
| gptkbp:license |
gptkb:Apache_License_2.0
|
| gptkbp:notableUser |
gptkb:EleutherAI
Microsoft DeepSpeed |
| gptkbp:programmingLanguage |
gptkb:Python
|
| gptkbp:purpose |
training large transformer models
|
| gptkbp:relatedTo |
gptkb:OpenAI_GPT
gptkb:T5 gptkb:NVIDIA_NeMo gptkb:BERT |
| gptkbp:repository |
https://github.com/NVIDIA/Megatron-LM
|
| gptkbp:supports |
gptkb:T5
gptkb:BERT gptkb:GPT custom transformer architectures |
| gptkbp:usedFor |
natural language processing
language model pretraining fine-tuning large models |
| gptkbp:bfsParent |
gptkb:NeMo_Megatron
|
| gptkbp:bfsLayer |
7
|