Statements (28)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:model
gptkb:convolutional_neural_network |
| gptkbp:arXivID |
gptkb:2004.05150
|
| gptkbp:attentionComplexity |
linear
|
| gptkbp:author |
gptkb:Arman_Cohan
gptkb:Iz_Beltagy gptkb:Matthew_E._Peters |
| gptkbp:availableOn |
gptkb:Hugging_Face_Transformers
|
| gptkbp:basedOn |
Transformer architecture
|
| gptkbp:citation |
high (hundreds to thousands)
|
| gptkbp:contrastsWith |
gptkb:BERT
gptkb:RoBERTa |
| gptkbp:designedFor |
long document processing
|
| gptkbp:developedBy |
gptkb:Allen_Institute_for_AI
|
| gptkbp:improves |
scalability for long sequences
|
| gptkbp:introducedIn |
2020
|
| gptkbp:language |
English
|
| gptkbp:length |
up to 4096 tokens (default)
|
| gptkbp:openSource |
yes
|
| gptkbp:publishedIn |
gptkb:arXiv
|
| gptkbp:supports |
question answering
summarization document classification |
| gptkbp:uses |
sparse attention mechanism
|
| gptkbp:bfsParent |
gptkb:Transformer_models
gptkb:Hugging_Face_models |
| gptkbp:bfsLayer |
7
|
| https://www.w3.org/2000/01/rdf-schema#label |
Longformer
|