Statements (55)
| Predicate | Object |
|---|---|
| gptkbp:instance_of |
gptkb:software
|
| gptkbp:bfsLayer |
4
|
| gptkbp:bfsParent |
gptkb:Hugging_Face
|
| gptkbp:developed_by |
gptkb:Microsoft
|
| gptkbp:enables |
efficient training of transformer models
|
| gptkbp:enhances |
training speed
inference performance |
| gptkbp:features |
checkpointing
gradient clipping mixed precision training Ze RO optimization activation checkpointing automatic mixed precision gradient accumulation tensor slicing |
| gptkbp:has |
gptkb:document
open-source license |
| https://www.w3.org/2000/01/rdf-schema#label |
Deep Speed
|
| gptkbp:improves |
gptkb:resource_utilization
scalability |
| gptkbp:integrates_with |
gptkb:Py_Torch
|
| gptkbp:is |
flexible
scalable widely adopted efficient user-friendly robust community-driven performance-oriented highly configurable actively maintained designed for high-performance computing designed for large-scale models part of the AI ecosystem |
| gptkbp:is_available_on |
gptkb:archive
|
| gptkbp:is_compatible_with |
AMDGP Us
NVIDIAGP Us |
| gptkbp:is_optimized_for |
large models
|
| gptkbp:is_used_by |
gptkb:physicist
gptkb:software data scientists |
| gptkbp:is_used_for |
deep learning
|
| gptkbp:language |
gptkb:Library
|
| gptkbp:provides |
memory efficiency
model parallelism dynamic loss scaling |
| gptkbp:reduces |
training costs
|
| gptkbp:released_in |
gptkb:2020
|
| gptkbp:supports |
gptkb:GPT-2
gptkb:GPT-3 gptkb:BERT gptkb:T5 distributed training multi-GPU training multi-node training |