Statements (57)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
Transformer model
|
| gptkbp:completed |
state-of-the-art results
|
| gptkbp:designedFor |
long documents
|
| gptkbp:developedBy |
gptkb:Facebook_AI_Research
|
| gptkbp:hasRelatedPatent |
question answering
language modeling summarization text classification |
| gptkbp:hasVariants |
Longformer-Decoder
Longformer-Encoder |
| https://www.w3.org/2000/01/rdf-schema#label |
Longformer
|
| gptkbp:improves |
scalability
|
| gptkbp:introduced |
2020
|
| gptkbp:isAttendedBy |
industry
academia |
| gptkbp:isAvailableIn |
Hugging Face Model Hub
|
| gptkbp:isBasedOn |
BERT architecture
|
| gptkbp:isCompatibleWith |
gptkb:PyTorch
TensorFlow |
| gptkbp:isConsidered |
a_breakthrough_in_NLP
|
| gptkbp:isDiscussedIn |
workshops
AI conferences |
| gptkbp:isDocumentedIn |
research papers
technical blogs |
| gptkbp:isEvaluatedBy |
gptkb:GLUE_benchmark
SQuAD benchmark Long Range Arena |
| gptkbp:isInfluencedBy |
Attention is All You Need
|
| gptkbp:isKnownFor |
enhancing performance
reducing computational cost handling long-range dependencies |
| gptkbp:isLocatedIn |
Python
|
| gptkbp:isOpenTo |
true
|
| gptkbp:isOptimizedFor |
memory efficiency
|
| gptkbp:isPartOf |
Big Bird
Reformer AI research community standard Transformers transformer family |
| gptkbp:isSupportedBy |
community contributions
|
| gptkbp:isTrainedIn |
large datasets
|
| gptkbp:isUsedBy |
developers
researchers |
| gptkbp:isUsedIn |
natural language processing tasks
|
| gptkbp:isUtilizedIn |
data analysis
language translation dialog systems sentiment analysis chatbots information retrieval text mining virtual assistants content generation knowledge extraction |
| gptkbp:mayHave |
up to 4096 tokens
|
| gptkbp:supports |
self-attention
|
| gptkbp:uses |
sparse attention mechanism
|