Statements (23)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:convolutional_neural_network
|
| gptkbp:availableOn |
gptkb:GitHub
|
| gptkbp:developedBy |
gptkb:NVIDIA
|
| gptkbp:enables |
multi-GPU training
multi-node training scaling transformer models |
| gptkbp:firstReleased |
2019
|
| https://www.w3.org/2000/01/rdf-schema#label |
NVIDIA Megatron
|
| gptkbp:integratesWith |
gptkb:PyTorch
|
| gptkbp:license |
gptkb:Apache_License_2.0
|
| gptkbp:optimizedFor |
gptkb:NVIDIA_GPUs
|
| gptkbp:supports |
distributed training
data parallelism model parallelism GPT models BERT models T5 models |
| gptkbp:usedBy |
gptkb:industry
gptkb:researchers |
| gptkbp:usedFor |
training large language models
|
| gptkbp:writtenBy |
gptkb:Python
|
| gptkbp:bfsParent |
gptkb:NVIDIA_NeMo
|
| gptkbp:bfsLayer |
6
|