# Why can't the gradient descent optimize a quantile regression loss?

Ok, so this may be a silly question, I know the literature recommends at length the simplex or interior point method to optimize a quantile regression problem. However I cannot figure out why a simple (stochastic or not) gradient descent isn't able to minimize the quantile loss which as a reminder looks like this.

Of course the gradient isn't defined at 0, but we can set it to 0 and in real life on large datasets with numerous dimensions, especially considering Doubles are rarely precisely equal with enough precision, it seems like a non issue.

I decided to experiment on a dataset and indeed it doesn't seem to work... I just can't figure out an intuition as to why. Can anybody help?

Follow up question, does anyone know of a way/library to fit a quantile regression with an easily distributable algorithm?

• Re parallelization - this package, cran.r-project.org/web/packages/cqrReg/index.html, presents an ADMM algo to solve quantile regression, and the ADMM algo can be parallelized... Feb 21, 2019 at 13:31
• Note that there are also smooth approximations of the quantile regression objective function that enable one to use gradient descent - see link.springer.com/article/10.1007/s13042-011-0031-2 and R package github.com/KarimOualkacha/cdaSQR Feb 21, 2019 at 13:52
• Feb 21, 2019 at 13:53
• I realize this is an old question, but... Did you try diminishing step sizes?
– bean
Aug 16, 2019 at 14:11

The intuition is as you say there is no gradient at $0$ for the quantile loss. Therefore any sub-gradient that you pick will be unstable. In general the way around this is to construct a Minorizing function and use an MM algorithm to fit the quantile regression. As far as I can tell spark.mllib does not have quantile regression support as of now.