What is the quantile covariance?

Question

Suppose that $X$ is a p-dimensional random vector and $Y$ is a random scalar. Then, Dodge and Whittaker (2009) indicate that the covariance of these two variables can be formulated as a minimization problem:

\begin{equation} \text{Cov}(Y,X)^T=\arg\inf_{\alpha, \beta}\{\mathbb{E}(Y-\alpha-\beta^T\text{Var}(X)^{-1}[X-\mathbb{E}(X)])^2\} \end{equation}

And based on this definition of the covariance they propose a quantile covariance defined for the $\tau^{th}$ quantile as:

\begin{equation} \text{Cov}_\tau(Y,X)^T=\arg\inf_{\alpha, \beta}\{\mathbb{E}\rho_\tau(Y-\alpha-\beta^T\text{Var}(X)^{-1}[X-\mathbb{E}(X)])\} \end{equation}

where $\rho_\tau(\cdot)$ is the check function for quantile regression defined by Koenker and Basset (1978).

I am trying to understand the way this quantile covariance works, but I am having problems from the very beginning, since it is based on a definition for the covariance that I have never seen before. So my questions are:

1. How is the covariance between a random scalar and a random vector calculated if the dimensions do not match?

2. Where is this definition as an optimization problem for the covariance coming from?

3. Any insights that help understanding the quantile covariance.

References:

• Dodge, Y. and Whittaker, J. (2009). Partial quantile regression. Metrika, 70:35–57.

• Koenker, R. and Bassett, G. (1978). Regression Quantiles. Econometrica, 46(1):33–50.

Answer 1

Setup

We'll assume that the feature covariance $\mathrm{var}(X) \in \mathbb{R}^{p \times p}$ is full rank. As you have in the original post, define $$(\hat{\alpha}, \hat{\beta}) = \arg\min_{\alpha, \beta} \mathbb{E} \left[ \left(Y - \left\{ \alpha + \beta^T \Sigma_{XX}^{-1} (X - \mu_X) \right\} \right)^2 \right]$$ and $$(\tilde{\alpha}, \tilde{\beta}) = \arg\min_{\alpha, \beta} \mathbb{E} \left[ \rho_\tau \left(Y - \left\{ \alpha + \beta^T \Sigma_{XX}^{-1} (X - \mu_X) \right\} \right) \right],$$ where e.g. $\mu_Y = \mathbb{E}[Y]$, $\Sigma_{XX} = \mathrm{cov}(X, X) = \mathrm{var}(X)$, and $\Sigma_{YX} = \mathrm{cov}(Y,X)$. Recall that the covariance $\Sigma_{XY} = \left(\Sigma_{YX}\right)^T$ and that the covariance $\Sigma_{XY}$ is itself just a vector whose $j^{\mathrm{th}}$ entry is the (scalar) covariance between $X_j$ and $Y$.

Review of OLS

Write that $$(\hat{u}, \hat{v}) = \arg\min_{u, v} \mathbb{E} \left[ \left(Y - \left\{ u + v^T (X - \mu_X) \right\} \right)^2 \right],$$ where $\hat{u} \in \mathbb{R}$ is the population-level OLS intercept and $\hat{v} \in \mathbb{R}^p$ is the population-level OLS slope, and so $\hat{u} + \hat{v}^T (X - \mu_X)$ is the population-level OLS line-of-best-fit.

Connection between OLS and covariance

Using matrix algebra, we could derive that $\hat{u} = \mu_Y$ and $$\hat{v}^T = \Sigma_{YX}\Sigma_{XX}^{-1}. \tag{*}$$ In terms of your first displayed equation, we see that $\hat{u} = \hat{\alpha}$ and $\hat{v} = \Sigma_{XX}^{-1}\,\hat{\beta}$; that is, $$\hat{\beta} = \Sigma_{XX} \, \hat{v} = \Sigma_{XX} \left( \Sigma_{XX}^{-1}\,\Sigma_{XY} \right) = \Sigma_{XY},$$ the covariance.

Quantile covariance

In order to define the "quantile covariance" $\Sigma_{YX}^{\mathrm{QUANTILE}}$, the authors continue to treat $(*)$ as true while switching away from a squared error loss function. Specifically, they could consider $$(\tilde{u}, \tilde{v}) = \arg\min_{u, v} \mathbb{E} \left[ \rho_\tau \left(Y - \left\{ u + v^T (X - \mathbb{E}[X]) \right\} \right) \right],$$ which is analogous to the population-level OLS fit earlier except now with quantile loss instead of squared loss. At this point, the goal is to connect the population-level quantile slope $\tilde{v}$ to a "quantile covariance". To do this, we could define that $$\tilde{v}^T =: \Sigma_{YX}^{\mathrm{QUANTILE}}\,\Sigma_{XX}^{-1}, \tag{**}$$ so that $\Sigma_{YX}^{\mathrm{QUANTILE}}$ is recovered through $\beta$ in your second displayed equation, i.e. $\tilde{\beta} = \Sigma_{XY}^{\mathrm{QUANTILE}}$.