BERT

E457858

BERT variant language model neural network model transformer-based model

BERT is a widely used transformer-based language model developed by Google that learns deep bidirectional representations of text for tasks like question answering and text classification.

Try in SPARQL Jump to: Surface forms Disambiguation Statements Elicitation Referenced by

All labels observed (3)

Label	Occurrences
BERT canonical	17
BERT base uncased	1
BERT_LARGE	1

How this entity was disambiguated

This entity first appeared as the object of triple T4651120 — resolving that mention is where its identity was fixed. The disambiguator weighed these candidate entities and picked the highlighted one (or “None”, minting a new entity). This is how homonymy is resolved: the same surface form can point to different entities.

NED1 Entity disambiguation (via context triple) gpt-5-mini-2025-08-07

Target entity: BERT
Context triple: [Transformer, foundationFor, BERT]

A. DistilBERT
DistilBERT is a smaller, faster, and lighter-weight distilled version of the BERT language model designed to retain most of its performance while being more efficient for practical NLP applications.
B. RoBERTa
RoBERTa is a robustly optimized transformer-based language model developed by Facebook AI that improves upon BERT through enhanced training strategies and larger-scale data.
C. DeBERTa
DeBERTa is a transformer-based language model developed by Microsoft that improves upon BERT and RoBERTa using disentangled attention and enhanced mask decoder mechanisms for superior natural language understanding.
D. GPT-2
GPT-2 is a large transformer-based language model known for generating coherent, human-like text and sparking widespread discussion about the implications of advanced AI text generation.
E. XLNet
XLNet is a generalized autoregressive pretraining model for natural language processing that improves on BERT by leveraging permutation-based language modeling to better capture bidirectional context.
F. None of above. chosen
G. Unsure - the case is ambiguous/there is not enough information to decide.

NED2 Entity disambiguation (via description) gpt-5-mini-2025-08-07

Target entity: BERT
Target entity description: BERT is a widely used transformer-based language model developed by Google that learns deep bidirectional representations of text for tasks like question answering and text classification.

A. DistilBERT
DistilBERT is a smaller, faster, and lighter-weight distilled version of the BERT language model designed to retain most of its performance while being more efficient for practical NLP applications.
B. RoBERTa
RoBERTa is a robustly optimized transformer-based language model developed by Facebook AI that improves upon BERT through enhanced training strategies and larger-scale data.
C. DeBERTa
DeBERTa is a transformer-based language model developed by Microsoft that improves upon BERT and RoBERTa using disentangled attention and enhanced mask decoder mechanisms for superior natural language understanding.
D. GPT-2
GPT-2 is a large transformer-based language model known for generating coherent, human-like text and sparking widespread discussion about the implications of advanced AI text generation.
E. XLNet
XLNet is a generalized autoregressive pretraining model for natural language processing that improves on BERT by leveraging permutation-based language modeling to better capture bidirectional context.
F. None of above. chosen

Statements (54)

Predicate	Object
instanceOf	BERT variant ⓘ BERT variant ⓘ language model ⓘ neural network model ⓘ transformer-based model ⓘ
acronymFor	Bidirectional Encoder Representations from Transformers ⓘ
architecture	Transformer ⓘ
benchmarkPerformance	state-of-the-art on GLUE at time of publication ⓘ state-of-the-art on SQuAD at time of publication ⓘ
developer	Google ⓘ Google AI Language ⓘ
fineTuningApproach	task-specific output layer on top of shared encoder ⓘ
fullName	Bidirectional Encoder Representations from Transformers ⓘ
hiddenSize	1024 ⓘ 768 ⓘ
implementation	TensorFlow ⓘ
influenced	ALBERT ⓘ DistilBERT ⓘ ELECTRA ⓘ RoBERTa ⓘ XLNet ⓘ
inputEmbedding	sum of token, segment, and position embeddings ⓘ
inputRepresentation	subword tokens ⓘ
introducedInPaper	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding ⓘ linked to: Bidirectional Encoder Representations from Transformers
language	English ⓘ
license	Apache License 2.0 ⓘ
numAttentionHeads	12 ⓘ 16 ⓘ
numLayers	12 ⓘ 24 ⓘ
openSource	true ⓘ
paperAuthors	Jacob Devlin ⓘ Kenton Lee ⓘ Kristina Toutanova ⓘ Ming-Wei Chang ⓘ
pretrainingObjective	masked language modeling ⓘ next sentence prediction ⓘ
publicationYear	2018 ⓘ
representationType	deep bidirectional contextual representations ⓘ
supportsTask	named entity recognition ⓘ natural language inference ⓘ paraphrase detection ⓘ question answering ⓘ semantic similarity ⓘ sentiment analysis ⓘ sequence labeling ⓘ text classification ⓘ token classification ⓘ
taskType	self-supervised learning ⓘ
trainingCorpus	BooksCorpus ⓘ linked to: BookCorpus English Wikipedia ⓘ
usesTokenization	WordPiece ⓘ
variant	BERT_BASE ⓘ BERT_LARGE ⓘ linked to: BERT

How these facts were elicited

Referenced by (19)

Full triples — surface form annotated when it differs from this entity's canonical label.

Transformer → foundationFor → BERT ⓘ

Layer Normalization → usedIn → BERT ⓘ

Elmo → influenced → BERT ⓘ

RoBERTa → basedOn → BERT ⓘ

RoBERTa → improvesUpon → BERT ⓘ

DistilBERT → basedOn → BERT ⓘ

DistilBERT → distilledFrom → BERT base uncased ⓘ

linked to: BERT

XLNet → comparedTo → BERT ⓘ

XLNet → improvesOn → BERT ⓘ

BART → combinesIdeasFrom → BERT ⓘ

ALBERT → basedOn → BERT ⓘ

DeBERTa → improvesUpon → BERT ⓘ

Longformer → comparedTo → BERT ⓘ

AutoConfig → supportsModelFamily → BERT ⓘ

Attention Is All You Need → inspiredModel → BERT ⓘ

Transformer encoder-only → relatedModel → BERT ⓘ

BERT → variant → BERT_LARGE ⓘ

linked to: BERT

Deep contextualized word representations → influenced → BERT ⓘ

Embeddings from Language Models → influenced → BERT ⓘ

All labels observed (3)

How this entity was disambiguated Show

Statements (54)

How these facts were elicited Show

Referenced by (19)

How this entity was disambiguated

How these facts were elicited