I am confused on the definitions of steepest descent.
- In some literature, such as this and this, steepest descent means using negative gradient direction and exact line search on that direction.
- But in this note, It seems as well as we are following negative gradient, the method can be called steepest descent.
Which one is correct? Can we call using fixed alpha (without line search) in negative gradient direction steepest descent?
Is the term "steepest descent" loosely defined?