5
$\begingroup$

Suppose that $X$ is a p-dimensional random vector and $Y$ is a random scalar. Then, Dodge and Whittaker (2009) indicate that the covariance of these two variables can be formulated as a minimization problem:

\begin{equation} \text{Cov}(Y,X)^T=\arg\inf_{\alpha, \beta}\{\mathbb{E}(Y-\alpha-\beta^T\text{Var}(X)^{-1}[X-\mathbb{E}(X)])^2\} \end{equation}

And based on this definition of the covariance they propose a quantile covariance defined for the $\tau^{th}$ quantile as:

\begin{equation} \text{Cov}_\tau(Y,X)^T=\arg\inf_{\alpha, \beta}\{\mathbb{E}\rho_\tau(Y-\alpha-\beta^T\text{Var}(X)^{-1}[X-\mathbb{E}(X)])\} \end{equation}

where $\rho_\tau(\cdot)$ is the check function for quantile regression defined by Koenker and Basset (1978).

I am trying to understand the way this quantile covariance works, but I am having problems from the very beginning, since it is based on a definition for the covariance that I have never seen before. So my questions are:

  1. How is the covariance between a random scalar and a random vector calculated if the dimensions do not match?

  2. Where is this definition as an optimization problem for the covariance coming from?

  3. Any insights that help understanding the quantile covariance.

References:

  • Dodge, Y. and Whittaker, J. (2009). Partial quantile regression. Metrika, 70:35–57.

  • Koenker, R. and Bassett, G. (1978). Regression Quantiles. Econometrica, 46(1):33–50.

$\endgroup$
  • 1
    $\begingroup$ The first minimization problem seems to me not to make sense. Is there not a typo? $\endgroup$ – Jesper Hybel Sep 22 at 13:26
  • $\begingroup$ Sorry, you were correct, I forgot squaring it. Now it is correct. You can find the formula in the appendix of the Partial quantile regression paper. $\endgroup$ – Álvaro Méndez Civieta Sep 23 at 6:32
3
+25
$\begingroup$

Setup

We'll assume that the feature covariance $\mathrm{var}(X) \in \mathbb{R}^{p \times p}$ is full rank. As you have in the original post, define $$(\hat{\alpha}, \hat{\beta}) = \arg\min_{\alpha, \beta} \mathbb{E} \left[ \left(Y - \left\{ \alpha + \beta^T \Sigma_{XX}^{-1} (X - \mu_X) \right\} \right)^2 \right]$$ and $$(\tilde{\alpha}, \tilde{\beta}) = \arg\min_{\alpha, \beta} \mathbb{E} \left[ \rho_\tau \left(Y - \left\{ \alpha + \beta^T \Sigma_{XX}^{-1} (X - \mu_X) \right\} \right) \right],$$ where e.g. $\mu_Y = \mathbb{E}[Y]$, $\Sigma_{XX} = \mathrm{cov}(X, X) = \mathrm{var}(X)$, and $\Sigma_{YX} = \mathrm{cov}(Y,X)$. Recall that the covariance $\Sigma_{XY} = \left(\Sigma_{YX}\right)^T$ and that the covariance $\Sigma_{XY}$ is itself just a vector whose $j^{\mathrm{th}}$ entry is the (scalar) covariance between $X_j$ and $Y$.

Review of OLS

Write that $$(\hat{u}, \hat{v}) = \arg\min_{u, v} \mathbb{E} \left[ \left(Y - \left\{ u + v^T (X - \mu_X) \right\} \right)^2 \right],$$ where $\hat{u} \in \mathbb{R}$ is the population-level OLS intercept and $\hat{v} \in \mathbb{R}^p$ is the population-level OLS slope, and so $\hat{u} + \hat{v}^T (X - \mu_X)$ is the population-level OLS line-of-best-fit.

Connection between OLS and covariance

Using matrix algebra, we could derive that $\hat{u} = \mu_Y$ and $$\hat{v}^T = \Sigma_{YX}\Sigma_{XX}^{-1}. \tag{*}$$ In terms of your first displayed equation, we see that $\hat{u} = \hat{\alpha}$ and $\hat{v} = \Sigma_{XX}^{-1}\,\hat{\beta}$; that is, $$\hat{\beta} = \Sigma_{XX} \, \hat{v} = \Sigma_{XX} \left( \Sigma_{XX}^{-1}\,\Sigma_{XY} \right) = \Sigma_{XY},$$ the covariance.

Quantile covariance

In order to define the "quantile covariance" $\Sigma_{YX}^{\mathrm{QUANTILE}}$, the authors continue to treat $(*)$ as true while switching away from a squared error loss function. Specifically, they could consider $$(\tilde{u}, \tilde{v}) = \arg\min_{u, v} \mathbb{E} \left[ \rho_\tau \left(Y - \left\{ u + v^T (X - \mathbb{E}[X]) \right\} \right) \right],$$ which is analogous to the population-level OLS fit earlier except now with quantile loss instead of squared loss. At this point, the goal is to connect the population-level quantile slope $\tilde{v}$ to a "quantile covariance". To do this, we could define that $$\tilde{v}^T =: \Sigma_{YX}^{\mathrm{QUANTILE}}\,\Sigma_{XX}^{-1}, \tag{**}$$ so that $\Sigma_{YX}^{\mathrm{QUANTILE}}$ is recovered through $\beta$ in your second displayed equation, i.e. $\tilde{\beta} = \Sigma_{XY}^{\mathrm{QUANTILE}}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.