# Gradient descent oscillating a lot. Have I chosen my step direction incorrectly?

I'm trying to run a basic gradient descent algorithm with a absolute loss function. I can get it to converge to a good solution by it requires a much lower step size and more iterations than had I used square loss. Is this normal? Should I expect absolute loss to take a longer time to come to a good solution or potentially oscillate around a solution more than say squared loss?

• Be careful in comparing $f() + \lambda \|x\|_1$ vs. $f() + \lambda \|x\|_2$: the norm1 and norm2 terms scale very differently, $N$ vs. $\sqrt N$. It's a good idea to look at the two terms separately (Funcmon), to see if they're reasonable. – denis Dec 16 '15 at 10:03
• A newer relevant post: stats.stackexchange.com/questions/260504/… – kjetil b halvorsen Oct 2 '18 at 12:48