Statements (24)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
mathematical optimization
|
| gptkbp:alsoKnownAs |
Polyak's heavy ball method
|
| gptkbp:application |
gptkb:machine_learning
gptkb:signal_processing control theory |
| gptkbp:category |
first-order optimization method
|
| gptkbp:citation |
Polyak, B.T. (1964). Some methods of speeding up the convergence of iteration methods. USSR Computational Mathematics and Mathematical Physics, 4(5), 1-17.
|
| gptkbp:convergesTo |
faster than gradient descent for some problems
|
| gptkbp:features |
momentum term
|
| https://www.w3.org/2000/01/rdf-schema#label |
Heavy ball method
|
| gptkbp:inspiredBy |
physical analogy of a ball rolling on a surface
|
| gptkbp:limitation |
may oscillate for large momentum
sensitive to parameter choice |
| gptkbp:parameter |
learning rate (\\alpha)
momentum coefficient (\\beta) |
| gptkbp:proposedBy |
gptkb:Boris_Polyak
|
| gptkbp:relatedTo |
gptkb:Nesterov_accelerated_gradient
gradient descent |
| gptkbp:updateRule |
x_{k+1} = x_k - \\alpha \\nabla f(x_k) + \\beta (x_k - x_{k-1})
|
| gptkbp:usedFor |
convex optimization
non-convex optimization |
| gptkbp:yearProposed |
1964
|
| gptkbp:bfsParent |
gptkb:Boris_Polyak
|
| gptkbp:bfsLayer |
6
|