In gradient descent algorithm, the update rule of vector parameter is as follow:

$w&space;=&space;w&space;-&space;learning\_rate&space;*&space;\nabla&space;f$

From this formula, i think that the update rule only depends on the sign of the gradient. So why don't we just use arbitrary vector instead of gradient vector in this formula.

• So you know how far to step in each direction. Different dimensions may need less or more changes. May 19 at 7:29
• @AryaMcCarthy Sorry can you make it rigorously ? It doesn't seem make sense to me. May 19 at 7:41

If you use an arbitrary vector, you could end up choosing $$-\nabla f$$ as well, and increase the value of the function.