Modeling Expected Value Using Quantile Regression as an Ensemble I'm trying to find a primer on a topic that I'm sure must have been studied, but I can't find anything on. Suppose we'd like to do regression in a supervised learning setting to learn the expected value of $y$ given $x$ i.e. $\mathbb{E}_{p(y|x)}[y]$. Rather than learning the expected value directly, suppose I instead perform quantile regression to learn $Q$ uniformly distributed quantiles and then combine the quantiles to estimate the desired expected value. Has anyone studied this algorithm? Does it have a name? Is there an advantage to be had by taking this circuitous route? For instance, each quantile may be biased, but their combination might have smaller variance?
 A: This is known as model-averaged quantile regression. It was discussed in Koenker "A note on L-estimates for linear models" (1984), though its origins stem from as early as the seminal paper of Koenker & Bassett "Regression quantiles" (1978). The 1984 paper actually proposes composite quantile regression, a natural competitor/counterpart of model-averaged quantile regression; the credit apparently goes to R.V.Hogg rather than Koenker himself.
Both types of quantile regression are discussed quite extensively in Koenker's monograph "Quantile Regression" (2005); for the model-averaged one, see Chapter 5. There is actually an equivalence between the two if unequal weights are allowed for. A recent comparison between the two estimators is provided in Bloznelis et al. "Composite versus model-averaged quantile regression" (2019).
It has been established that the model-averaged as well as the composite estimator can beat the OLS estimator of $\beta$ in a linear regression model in terms of asymptotic efficiency for certain error distributions. Actually, the composite estimator with equal weights has some nice efficiency guarantees; given some assumptions, the asymptotic relative efficiency of the composite quantile estimator with equal weights is at least ~70% that of an OLS estimator, but it can be much more efficient than it (Zou & Yuan "Composite quantile regression and the oracle model selection theory" (2008)).
