|
gptkbp:instance_of
|
gptkb:Natural_Language_Processing
gptkb:language
|
|
gptkbp:application
|
natural language understanding
question answering
sentiment analysis
|
|
gptkbp:architecture
|
gptkb:Transformers
|
|
gptkbp:available_in
|
gptkb:Tensor_Flow
gptkb:Hugging_Face_Transformers
gptkb:Py_Torch
|
|
gptkbp:based_on
|
gptkb:Transformer_Architecture
|
|
gptkbp:community_support
|
high
|
|
gptkbp:developed_by
|
gptkb:Google
|
|
gptkbp:evaluates
|
gptkb:historical_memory
F1 score
accuracy
precision
|
|
gptkbp:fine_tuning_method
|
gptkb:Adam_optimizer
gradient descent
|
|
gptkbp:has_achieved
|
gptkb:SQu_AD_Benchmark
gptkb:GLUE_Benchmark
state-of-the-art results
|
|
gptkbp:has_applications_in
|
gptkb:customer_service
gptkb:Education
Finance
Healthcare
Legal
|
|
gptkbp:has_function
|
110 million
345 million (large model)
|
|
gptkbp:has_limitations
|
May Overfit Small Datasets
Not Always Interpretable
Requires Large Compute Resources
Sensitive to Input Quality
|
|
gptkbp:has_publications
|
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
|
|
gptkbp:has_variants
|
gptkb:BERTweet
gptkb:XLM-Ro_BERTa
gptkb:Distil_BERT
gptkb:Bio_BERT
gptkb:Clinical_BERT
gptkb:Tiny_BERT
gptkb:m_BERT
gptkb:Ro_BERTa
gptkb:BERT-Base
gptkb:BERT-Large
ALBERT
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
BERT
|
|
gptkbp:impact
|
gptkb:significant
|
|
gptkbp:improves
|
gptkb:Biology
Named Entity Recognition
Question Answering
question answering
sentiment analysis
named entity recognition
|
|
gptkbp:influenced_by
|
gptkb:ELMo
gptkb:GPT
|
|
gptkbp:input_output
|
contextual embeddings
tokenized text
|
|
gptkbp:introduced_in
|
gptkb:2018
|
|
gptkbp:is_applied_in
|
language translation
text summarization
text classification
|
|
gptkbp:is_available_on
|
gptkb:Hugging_Face_Transformers
gptkb:Py_Torch_Hub
gptkb:Tensor_Flow_Hub
|
|
gptkbp:is_based_on
|
self-attention mechanism
encoder-decoder architecture
|
|
gptkbp:is_cited_in
|
Conferences
Industry Applications
Numerous Research Papers
Academic Publications
over 10000
|
|
gptkbp:is_evaluated_by
|
Accuracy
F1 Score
Precision
Recall
|
|
gptkbp:is_known_for
|
contextual embeddings
bidirectional training
transfer learning in NLP
|
|
gptkbp:is_open_source
|
gptkb:True
gptkb:true
|
|
gptkbp:is_part_of
|
NLP Research Community
|
|
gptkbp:is_popular_in
|
gptkb:scientific_community
industry applications
AI development
|
|
gptkbp:is_related_to
|
gptkb:Natural_Language_Processing
gptkb:Artificial_Intelligence
gptkb:machine_learning
gptkb:Deep_Learning
Text Mining
Semantic Analysis
|
|
gptkbp:is_scalable
|
Multiple Languages
Large Datasets
Domain-Specific Tasks
|
|
gptkbp:is_tasked_with
|
natural language processing
|
|
gptkbp:is_trained_in
|
gptkb:Wikipedia
gptkb:Book_Corpus
large text corpora
diverse datasets
|
|
gptkbp:is_used_in
|
gptkb:academic_research
gptkb:Google_Assistant
gptkb:Google
Language Translation
Sentiment Analysis
Text Summarization
chatbots
virtual assistants
social media analysis
customer support systems
content recommendation systems
|
|
gptkbp:language
|
gptkb:French
gptkb:Spanish
gptkb:German
Chinese
English
|
|
gptkbp:max_input_length
|
512 tokens
|
|
gptkbp:model
|
Base (110 M parameters)
Large (345 M parameters)
|
|
gptkbp:output_layer
|
softmax layer
|
|
gptkbp:performance
|
gptkb:GLUE
gptkb:Co_NLL-2003
gptkb:SQu_AD
gptkb:MNLI
gptkb:RACE
state-of-the-art
|
|
gptkbp:pre_training_task
|
Masked Language Model
Next Sentence Prediction
|
|
gptkbp:pretraining_task
|
masked language modeling
next sentence prediction
|
|
gptkbp:provides_information_on
|
gptkb:Wikipedia
gptkb:Book_Corpus
|
|
gptkbp:related_to
|
gptkb:NLP
deep learning
transfer learning
contextual embeddings
semantic understanding
|
|
gptkbp:release_date
|
gptkb:2018
|
|
gptkbp:successor
|
gptkb:Distil_BERT
gptkb:Tiny_BERT
gptkb:Ro_BERTa
ALBERT
|
|
gptkbp:supports
|
gptkb:stage_adaptation
fine-tuning
|
|
gptkbp:system_requirements
|
gptkb:TPUs
GPUs
|
|
gptkbp:tokenization
|
Word Piece
|
|
gptkbp:training
|
supervised learning
unsupervised learning
days to weeks
|
|
gptkbp:tuning
|
task-specific datasets
|
|
gptkbp:user_base
|
gptkb:developers
gptkb:researchers
data scientists
|
|
gptkbp:uses
|
Attention Mechanism
deep learning
attention mechanism
masked language modeling
Bidirectional Context
Word Piece Tokenization
next sentence prediction
|
|
gptkbp:bfsParent
|
gptkb:Transformers
gptkb:Google
|
|
gptkbp:bfsLayer
|
4
|