Search Direction Correction with Normalized Gradient Makes First-Order Methods Faster

The so-called fast inertial relaxation engine is a first-order method for unconstrained smooth optimization problems. It updates the search direction by a linear combination of the past search dire...