Training Recurrent Neural Networks
GPTKB entity
Statements (48)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
Machine Learning Task
|
| gptkbp:appliesTo |
Recurrent Neural Networks
|
| gptkbp:canBe |
Gated Recurrent Unit
Long Short-Term Memory |
| gptkbp:canBeAcceleratedBy |
GPU Computation
TPU Computation |
| gptkbp:canBeOptimizedWith |
gptkb:RMSprop
gptkb:Adam_Optimizer SGD |
| gptkbp:canBeParallelizedBy |
gptkb:Truncated_Backpropagation_Through_Time
|
| gptkbp:challenge |
Exploding Gradient Problem
Vanishing Gradient Problem |
| https://www.w3.org/2000/01/rdf-schema#label |
Training Recurrent Neural Networks
|
| gptkbp:implementedIn |
gptkb:TensorFlow
gptkb:Keras gptkb:MXNet gptkb:Theano gptkb:PyTorch |
| gptkbp:improves |
gptkb:Dropout
Batch Normalization Curriculum Learning Data Augmentation Early Stopping Gradient Clipping Layer Normalization Learning Rate Scheduling Regularization Scheduled Sampling Sequence Padding Teacher Forcing Weight Initialization |
| gptkbp:involves |
Backpropagation Through Time
Gradient Descent |
| gptkbp:monitors |
Accuracy
Perplexity Training Loss Validation Loss |
| gptkbp:requires |
Sequential Data
|
| gptkbp:suffersFrom |
Overfitting
|
| gptkbp:usedIn |
gptkb:Machine_Translation
gptkb:Natural_Language_Processing gptkb:Speech_Recognition Video Analysis Music Generation Text Generation Time Series Prediction |
| gptkbp:bfsParent |
gptkb:Ilya_Sutskever
|
| gptkbp:bfsLayer |
5
|