Statements (60)
| Predicate | Object |
|---|---|
| gptkbp:instance_of |
gptkb:Artificial_Intelligence
|
| gptkbp:can_be_combined_with |
momentum and RMSProp
|
| gptkbp:can_be_used_for |
stochastic optimization
|
| gptkbp:can_handle |
sparse gradients
|
| gptkbp:developed_by |
gptkb:2014
gptkb:D._P_Kingma |
| gptkbp:has_function |
beta1
beta2 epsilon learning rate |
| https://www.w3.org/2000/01/rdf-schema#label |
Adam optimizer
|
| gptkbp:improves |
SGD
|
| gptkbp:is_adaptive |
yes
|
| gptkbp:is_based_on |
first and second moments of gradients
|
| gptkbp:is_compared_to |
other optimizers
|
| gptkbp:is_considered_as |
state-of-the-art optimizer
|
| gptkbp:is_documented_in |
research papers
online tutorials technical blogs |
| gptkbp:is_effective_against |
non-stationary objectives
|
| gptkbp:is_evaluated_by |
convergence speed
real-world applications stability robustness empirical studies benchmark tests final accuracy |
| gptkbp:is_implemented_in |
gptkb:Tensor_Flow
gptkb:Keras gptkb:Py_Torch |
| gptkbp:is_less_sensitive_to |
initialization
|
| gptkbp:is_often_used_in |
computer vision
natural language processing |
| gptkbp:is_part_of |
deep learning frameworks
|
| gptkbp:is_popular_in |
gptkb:neural_networks
|
| gptkbp:is_recommended_by |
gptkb:Adagrad
gptkb:Adadelta RMSProp |
| gptkbp:is_related_to |
backpropagation
gradient descent loss function training process |
| gptkbp:is_robust_to |
noisy gradients
|
| gptkbp:is_used_for |
transfer learning
model training hyperparameter optimization feature learning fine-tuning models |
| gptkbp:is_used_in |
gptkb:machine_learning
deep learning reinforcement learning |
| gptkbp:speed |
traditional SGD
|
| gptkbp:suitable_for |
large datasets
online learning very small datasets highly oscillatory functions |
| gptkbp:tuning |
hyperparameters
|
| gptkbp:uses |
adaptive learning rates
|
| gptkbp:bfsParent |
gptkb:neural_networks
|
| gptkbp:bfsLayer |
4
|