Statements (30)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
large language model
|
| gptkbp:basedOn |
transformer architecture
|
| gptkbp:compatibleWith |
gptkb:DeepSpeed
gptkb:Megatron-LM gptkb:PyTorch |
| gptkbp:developedBy |
gptkb:EleutherAI
|
| gptkbp:documentation |
https://gpt-neox.readthedocs.io/
|
| gptkbp:firstReleased |
2022
|
| gptkbp:hasFeature |
efficient memory usage
customizable architecture scalable to large clusters |
| gptkbp:hasModelSize |
20 billion parameters
|
| gptkbp:hasPretrainedModels |
true
|
| https://www.w3.org/2000/01/rdf-schema#label |
GPT-NeoX
|
| gptkbp:latestReleaseVersion |
20B
|
| gptkbp:license |
gptkb:Apache_License_2.0
|
| gptkbp:openSource |
true
|
| gptkbp:repository |
https://github.com/EleutherAI/gpt-neox
|
| gptkbp:supports |
distributed training
data parallelism model parallelism |
| gptkbp:trainer |
gptkb:The_Pile
|
| gptkbp:usedBy |
gptkb:researchers
developers |
| gptkbp:usedFor |
natural language processing
text generation language modeling |
| gptkbp:writtenBy |
gptkb:Python
|
| gptkbp:bfsParent |
gptkb:transformation
|
| gptkbp:bfsLayer |
5
|