Granger Causality and positive semidefiniteness

Question

Suppose that we have vector $y_t = (z_t, x_t)'$ for $t = 1, 2, \dots, T$. Let $\Omega_t$ be the information set available at time $t$ and $z_t(h|\Omega_t)$ be optimal $h$-step predictor. $\Sigma_z(h|\Omega_t)$ denotes forecast MSE matrix.

We say that $x_t$ helps to predict $z_t$ (Granger Causality) if $$\Sigma_z\left(h|\Omega_t\setminus\{x_s|s\leq t\}\right) - \Sigma_z(h|\Omega_t)$$ for at least one $h = 1,2, \dots$ is positive semidefinite and $$\Sigma_z(h|\Omega_t) \neq \Sigma_z\left(h|\Omega_t\setminus\{x_s|s\leq t\}\right).$$ The definition is clear for me. $x_t$ helps predicting $z_t$ if the covariance matrix of the prediction without using $x_t$ is bigger then when we use $x_t$ in a matrix sense.

My confusion is more about interpretation of the semidefinite matrix. By definition $m\times m$ symmetric matrix $A$ is positive semidefinite iff for any $m\times 1$ vector $z$ $$z'Az \geq 0.$$

Let $A \equiv \Sigma_z\left(h|\Omega_t\setminus\{x_s|s\leq t\}\right) - \Sigma_z(h|\Omega_t).$ So what is the intuition that if $x'Ax \geq 0$, then I can conclude that $x_t$ helps in predicting $z_t$?

---

Edit: Just to clarify, the question is more about the concept that the variance of one random vector $x$ (denote it by $\Omega_x$) is greater than for the other random vector $y$ (denote it by $\Omega_y$) if the difference between their covariance matrices, $\Omega_x - \Omega_y$ is psd. As one more example, consider GLS and OLS estimators in a linear regression setting, $\hat\beta_{GLS}$ and $\hat\beta_{OLS}$ respectively. We say that the GLS estimator is more efficient if $B \equiv \Omega_{OLS} - \Omega_{GLS}$ is psd matrix. That is, for any vector $z$ it holds that $$z'Bz \geq 0.$$ In other words, covariance matrix $\Omega_{GLS}$ is smaller in a matrix sense than $\Omega_{OLS}$. What I am looking for is an explanation/intuition of why the case when the matrix $\Omega_{OLS} - \Omega_{GLS}$ is psd implies that the GLS estimator is more efficient then the OLS? Maybe another formulation: what does it mean for one matrix to be smaller then the other?

Answer 1

Positive semidefinitness of the difference of two matrices is the concept that is analogous to the difference being non-negative for unidimensional entities.

In particular, a positive semidifinitness matrix has all its diagonal elements non-negative. Diagonal elements here represent same-period forecast MSE's so positive definitness says that the forecast MSE without $x$ will be equal or greater compared to when we include $x$.

So this is a weak form of Granger Causality. We would want the difference of the two matrices to be positive definite in order to have strictly greater same-period MSE's

Answer 2

The following comes with the caveat that I know nothing about Granger causality, just about how positive semidefinite matrices relate to covariances of random variables. According to @jbowman in the comments, this observation is apparently correct regarding the motivation for using positive semidefinite matrices in the context of Granger causality.

1. I think your definition of positive semidefiniteness is missing the condition that the matrix $A$ needs to be symmetric (assuming $A$ has real entries -- the more general condition, when $A$ possibly has complex entries, is that $A$ be Herimitian). Although some authors disagree.

The analogy is with bilinear forms, for which questions of positive/negative semidefiniteness only make sense in the context of symmetric bilinear forms (I think -- since there is a one-to-one correspondence between symmetric bilinear forms and quadratic forms, see here).

Matrices $A$ can be used to represent both linear transformations and bilinear forms even though all three are different objects -- this is because they are all vector spaces of the same dimension, see here or here.

2. I bring up the likely implied requirement that the matrix $A$ be symmetric for the following reason:

I imagine the condition on $A$ being symmetric and positive semi-definite has something to do with the fact that any covariance matrix $A$ is symmetric and positive semidefinite, and any symmetric positive semidefinite matrix could represent the covariance matrix of some multivariate distribution. See, for example also (1)(2)(3)(4)(5)(6)(7) (from the first page of my Google search.)

Technically, the above only addresses one direction of my claim -- that any covariance matrix is positive-semidefinite. It does not address the other direction of my claim -- that any positive-semidefinite matrix is the covariance matrix for some random vector (it shouldn't necessarily be clear which, and in general there is no reason to believe the distribution will be unique.) For that, see here and Exercises 2-4 outlining a proof here.

Basically, the covariance matrices are symmetric because covariance is a symmetric relationship.

And they are positive semi-definite because the variance of a random variable is always non-negative and since symmetric matrices are always diagonalizable.

The converse holds because you can always "rig" a simple enough distribution (e.g. multivariate Gaussian) to give you a covariance matrix with the entries you want, as long as those entries constitute a symmetric, positive-semidefinite matrix.

---

Anyway, my guess is that the condition in the definition of Granger causality, that the matrix $A$ be positive semi-definite (and symmetric), probably has a probabilistic interpretation and motivation. The most likely such interpretation or motivation is that it is supposed to be a stand-in for an arbitrary covariance matrix, which is what positive semi-definite matrices are.

Your question also suggests this interpretation too:

$x_t$ helps predicting $z_t$ if the covariance matrix of the prediction without using $x_t$ is bigger then when we use $x_t$ in a matrix sense. ... So what is the intuition that if $x′Ax≥0$, then I can conclude that $x_t$ helps in predicting $z_t$?

This is just a guess, which is why I initially posted it as a comment, but I was encouraged to post it as an answer by another user, which suggests that my guess might in fact be correct.