Statements (27)
Predicate | Object |
---|---|
gptkbp:instanceOf |
gptkb:model
gptkb:convolutional_neural_network |
gptkbp:arXivID |
gptkb:2004.05150
|
gptkbp:attentionComplexity |
linear
|
gptkbp:author |
gptkb:Arman_Cohan
gptkb:Iz_Beltagy gptkb:Matthew_E._Peters |
gptkbp:availableOn |
gptkb:Hugging_Face_Transformers
|
gptkbp:basedOn |
Transformer architecture
|
gptkbp:citation |
high (hundreds to thousands)
|
gptkbp:contrastsWith |
gptkb:BERT
gptkb:RoBERTa |
gptkbp:designedFor |
long document processing
|
gptkbp:developedBy |
gptkb:Allen_Institute_for_AI
|
https://www.w3.org/2000/01/rdf-schema#label |
Longformer
|
gptkbp:improves |
scalability for long sequences
|
gptkbp:introducedIn |
2020
|
gptkbp:language |
English
|
gptkbp:length |
up to 4096 tokens (default)
|
gptkbp:openSource |
yes
|
gptkbp:publishedIn |
gptkb:arXiv
|
gptkbp:supports |
question answering
summarization document classification |
gptkbp:uses |
sparse attention mechanism
|
gptkbp:bfsParent |
gptkb:transformation
|
gptkbp:bfsLayer |
5
|