# Linear Regression and Quantile Regression

Linear regression using the method of least squares estimates the conditional mean of the response variable across values of the predictor variables.

Quantile regression estimates a conditional quantile of the response variable across values of the predictor variables.

The least squares method minimises the sum of squared residuals, and quantile regression minimises a loss function, $$\rho(\tau)$$, that depends on the quantile of interest.

I think I understand the methods.

My problem: I have a model for the failure times of a material, $$\log(T) \sim N(\mu(x),\sigma)$$, where $$\mu(x) = \beta_0 + \beta_1 x_1 +\beta x_2 + \dots + \beta_n x_n$$ is the mean, $$x_1, \dots, x_n$$ are covariates, and $$\sigma$$ is the standard deviation.

I usually approach a problem from a Bayesian perspective (well, I call myself a Bayesian but I often implement a model using Stan and uninformative priors so I am not sure if I am classed as a Bayesian).

I specified the likelihood and prior distributions and obtained estimates for the model parameters (conditional on the covariates). This worked fine. I am able to obtain any quantile of interest for the (log) failure times from the posterior distribution.

I was happy with this analysis until I started to overthink the problem.

"When you changed from linear regression to quantile regression you had to change the loss function. You have performed a Bayesian analysis like usual, you must adjust for quantile regression".

I think my above thoughts are incorrect. If least squares estimates provided parameter estimates for the log failure times, one could then obtain any estimate of interest. However, least squares estimates the conditional mean $$E[\log(T)]$$, and not $$\log(T)$$ itself. Therefore, I have to change the loss function of interest to $$\rho(\tau)$$, if I want the $$\tau$$ quantile of $$\log(T)$$. Another loss function would be required if I want the $$\tau^*$$ quantile. This is because these methods estimate quantities of interest only, and not the random variable of interest.

The Bayesian approach (I am not saying only a Bayesian approach can do this. ML estimates with bootstrap or something would also provide estimates with uncertainty for $$\log(T)$$. I am comparing a Bayesian approach to least squares and quantile regression.) estimates the distribution of $$\log(T)$$ and not of $$E[\log(T)]$$, and hence I do not need to adjust anything, like when going from linear regression to quantile regression.

Can anyone please confirm my understanding. I think I was just overthinking for a moment because I had to adjust the loss function when moving from linear regression to quantile regression and thought "a standard Bayesian approach must need to be adjusted when you're interested in quantile regression". I hope this makes sense.

What you have seems to be a fairly standard log-normal survival/reliability model of continuous-time failure data. You presumably didn't model quantiles directly, but rather the entire function describing $$\log (T)$$ as a function of covariates and time. The way you did this was with likelihood-based methods. That doesn't seem to be what is usually considered "quantile regression." Rather, it's a Bayesian survival analysis.

If a cumulative distribution of failure times conditional on covariates is $$F(t)$$, then the corresponding distribution of survival times is $$S(t)=1-F(t)$$. All you are doing to get quantiles of survival times is sampling from your posterior estimate of $$F(t)$$.

One caution: did you observe failure times for all samples of the material, or did some samples not fail at all? In the latter case, make sure that your model incorporated the contribution of such samples with right-censored survival times to the likelihood. Otherwise your estimates are likely to be biased.

• Thank you for the reply. I am happy with the analysis (ignoring censored observations as this post is more about the methods). My main question being that I have performed linear regression many times, and I have performed a Bayesian analysis many times. Both of which I understand. I recently learned about quantile regression and realised I had to change the loss function (to obtain the quantiles of interest). I then started to think, do I need to change anything about the Bayesian analysis if I am more interested in quantiles?
– JLee
Commented Feb 18, 2022 at 13:52
• I know some distributions can be parameterised in terms of quantiles. I think my brain was linking linear regression to the Bayesian analysis and then thinking if quantile regression (in some sense the quantile version of linear regression) requires the loss function to be changed then I must change the likelihood in some way during a Bayesian analysis for Bayesian quantile regression. This is probably my brain overthinking a problem I already understand. C.f. final three paragraphs of original post.
– JLee
Commented Feb 18, 2022 at 13:55
• I think I just wanted confirmation that two methods are needed (linear regression and quantile regression) when one is interested in the conditional mean and a conditional quantile, because these methods specifically set out to estimate quantities of interest ($E[\log(T)]$, and $\rho(\tau)$), whereas the Bayesian approach provides a posterior distribution for $\log(T)$ and hence any quantity of interest can be obtained (for example, the expectation or a quantile).
– JLee
Commented Feb 18, 2022 at 14:03
• Therefore, if you have the posterior distribution for $\log(T)$ no adjustments are needed unlike when moving from linear regression to quantile regression.
– JLee
Commented Feb 18, 2022 at 14:05
• @JLee you are correct. You should know that Bayesian quantile regression is not straightforward as there is no simple parametric likelihood; see for example this paper. With your log-normal survival analysis you have specified a parametric likelihood that covers the entire distribution of interest, providing a posterior from which you can sample at will.
– EdM
Commented Feb 18, 2022 at 15:41