|
gptkbp:instance_of
|
gptkb:language
|
|
gptkbp:bfsLayer
|
4
|
|
gptkbp:bfsParent
|
gptkb:BERT
|
|
gptkbp:application
|
language translation
text generation
text summarization
text classification
|
|
gptkbp:architectural_style
|
gptkb:Transformers_character
|
|
gptkbp:cache_size
|
gptkb:32
|
|
gptkbp:coat_of_arms
|
gptkb:24
|
|
gptkbp:community_support
|
high
|
|
gptkbp:data_usage
|
3.3 billion words
|
|
gptkbp:developed_by
|
gptkb:Job_Search_Engine
|
|
gptkbp:field_of_study
|
gptkb:API
gptkb:Deep_Learning
gptkb:NLP
|
|
gptkbp:first_introduced
|
Xavier initialization
|
|
gptkbp:focus
|
gptkb:16
|
|
https://www.w3.org/2000/01/rdf-schema#label
|
BERT-Large
|
|
gptkbp:impact
|
gptkb:significant
|
|
gptkbp:influenced_by
|
gptkb:GPT
EL Mo
|
|
gptkbp:input_output
|
contextual embeddings
softmax
tokenized text
|
|
gptkbp:is_a_framework_for
|
gptkb:Graphics_Processing_Unit
gptkb:Py_Torch
|
|
gptkbp:is_available_on
|
gptkb:Hugging_Face
gptkb:Py_Torch_Hub
gptkb:Tensor_Flow_Hub
|
|
gptkbp:is_compared_to
|
gptkb:BERT-Base
ALBERT
Ro BER Ta
XL Net
|
|
gptkbp:is_evaluated_by
|
gptkb:GLUE
gptkb:Co_NLL-2003
gptkb:MNLI
S Qu AD
|
|
gptkbp:is_open_source
|
gptkb:theorem
|
|
gptkbp:is_optimized_for
|
gptkb:Adam
|
|
gptkbp:is_tasked_with
|
natural language understanding
question answering
sentiment analysis
named entity recognition
|
|
gptkbp:language
|
English
|
|
gptkbp:length
|
512
|
|
gptkbp:losses
|
cross-entropy loss
|
|
gptkbp:orbital_period
|
345 million
|
|
gptkbp:performance
|
state-of-the-art
|
|
gptkbp:reduces
|
0.1
|
|
gptkbp:release_date
|
gptkb:2019
|
|
gptkbp:size
|
1024
|
|
gptkbp:successor
|
gptkb:BERT-Base
|
|
gptkbp:symbolizes
|
Word Piece
|
|
gptkbp:training
|
gptkb:Wikipedia
gptkb:Book_Corpus
supervised
unsupervised
2e-5
|
|
gptkbp:tuning
|
possible
|
|
gptkbp:uses
|
widely adopted
masked language modeling
next sentence prediction
|