BERT

GPTKB entity

Statements (160)

Predicate	Object
gptkbp:instance_of	gptkb:software gptkb:language
gptkbp:bfsLayer	3
gptkbp:bfsParent	gptkb:Job_Search_Engine gptkb:studio
gptkbp:allows	May Overfit Small Datasets Not Always Interpretable Requires Large Compute Resources Sensitive to Input Quality
gptkbp:application	natural language understanding question answering sentiment analysis
gptkbp:applies_to	language translation text summarization text classification
gptkbp:architectural_style	gptkb:Transformers_character
gptkbp:based_on	gptkb:Transformer_Architecture self-attention mechanism encoder-decoder architecture
gptkbp:community_support	high
gptkbp:developed_by	gptkb:Job_Search_Engine
gptkbp:has_achievements	gptkb:GLUE_Benchmark state-of-the-art results S Qu AD Benchmark
gptkbp:has_method	110 million 345 million (large model)
gptkbp:has_programs	gptkb:Educational_Institution gptkb:railway_line Finance Healthcare Legal
gptkbp:has_variants	gptkb:Distil_BERT gptkb:Bio_BERT gptkb:Clinical_BERT gptkb:Tiny_BERT gptkb:m_BERT gptkb:BERT-Base gptkb:BERT-Large ALBERT Ro BER Ta BER Tweet XLM-Ro BER Ta
https://www.w3.org/2000/01/rdf-schema#label	BERT
gptkbp:impact	gptkb:significant
gptkbp:improves	gptkb:computer Named Entity Recognition Question Answering question answering sentiment analysis named entity recognition
gptkbp:influenced_by	gptkb:GPT EL Mo
gptkbp:input_output	contextual embeddings 512 tokens tokenized text softmax layer
gptkbp:introduced	gptkb:2018
gptkbp:is_available_in	gptkb:Graphics_Processing_Unit gptkb:Hugging_Face_Transformers gptkb:Py_Torch
gptkbp:is_available_on	gptkb:Hugging_Face_Transformers gptkb:Py_Torch_Hub gptkb:Tensor_Flow_Hub
gptkbp:is_cited_in	Conferences Industry Applications Numerous Research Papers Academic Publications over 10000
gptkbp:is_evaluated_by	gptkb:municipality Accuracy F1 Score Precision Recall F1 score accuracy precision
gptkbp:is_known_for	contextual embeddings bidirectional training transfer learning in NLP
gptkbp:is_open_source	gptkb:theorem
gptkbp:is_part_of	NLP Research Community
gptkbp:is_popular_in	gptkb:Research_Institute industry applications AI development
gptkbp:is_related_to	gptkb:Artificial_Intelligence gptkb:software gptkb:software_framework gptkb:Deep_Learning Text Mining Semantic Analysis
gptkbp:is_scalable	Multiple Languages Large Datasets Domain-Specific Tasks
gptkbp:is_tasked_with	natural language processing
gptkbp:is_used_in	gptkb:musician gptkb:Job_Search_Engine gptkb:academic_research Language Translation Sentiment Analysis Text Summarization chatbots virtual assistants social media analysis customer support systems content recommendation systems
gptkbp:language	gptkb:French gptkb:Spanish gptkb:Native_American_tribe Chinese English Word Piece
gptkbp:performance	gptkb:GLUE gptkb:Co_NLL-2003 gptkb:MNLI gptkb:RACE state-of-the-art S Qu AD
gptkbp:provides_information_on	gptkb:Wikipedia gptkb:Book_Corpus
gptkbp:publishes	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
gptkbp:related_model	Base (110 M parameters) Large (345 M parameters)
gptkbp:related_to	gptkb:NLP deep learning transfer learning contextual embeddings semantic understanding
gptkbp:release_date	gptkb:2018
gptkbp:successor	gptkb:Distil_BERT gptkb:Tiny_BERT ALBERT Ro BER Ta
gptkbp:supports	gptkb:streaming_service fine-tuning
gptkbp:system_requirements	TP Us GP Us
gptkbp:training	gptkb:Wikipedia gptkb:Book_Corpus supervised learning unsupervised learning Masked Language Model Next Sentence Prediction large text corpora days to weeks diverse datasets masked language modeling next sentence prediction
gptkbp:tuning	gptkb:Adam_optimizer gradient descent task-specific datasets
gptkbp:user_base	gptkb:physicist gptkb:software data scientists
gptkbp:uses	Attention Mechanism deep learning attention mechanism masked language modeling Bidirectional Context Word Piece Tokenization next sentence prediction