1
$\begingroup$

Assume that we want to compare the forecast quality of various forecasters $f$ on $n$ values such as stock-market prices or whatever. We could then define a "Mahalanobis-Distance" (MD) (or rather Pseudo-Mahalanobis-Distance measure) for forecaster $f=1...F$ in year $j =1...J$ via: $(p_{f,j}-a_j)^T\hat{C}^{-1}(p_{f,j}-a_j)$. Where $p_{f,j}$ denotes the mean-vector of the predictions of forecaster $f$ from year $j-1$ for $j$ and $a_j$ denotes the vector that contains the actual values for the $n$ aggregates in year $j$. ($a_j$ is a realisation of the RV $A_j$ and $p_{f,j}$ is a realisation of the RV $P_{f,j}$ )

Furthermore $\hat{C} := \frac{1}{J-1}\sum_{j=1}^Ja_ja_j^T $. My question is which assumptions must be met in order for the above defiend "MD" to be meaningful. It seems to me that it must hold that $Cov(A_j)=Cov(P_{f,j}) \forall f=1...F $ and $j=1...J$

(in this case we should also use the realisations of the $P_{f,j}$ to estimate the common Covariane matrix, but at least the above defined metric still makes sense, even tough the estimate for the Covariance Matrix is not optimal.)

But in case 1: $Cov(A_j)=Cov(A_i) \forall i,j=1...J$ but $Cov(P_{f,j})\neq Cov(A_j)$ the above defined "MD" does not make sense to me at all. As far as I understand the purpose of multiplying with $C^{-1}$ is to normalise the RVs $P_{f,j}$ and $A_j$, which can be seen in the following equation:

$(P_{f,j}-A_j)^T)C^{-1}(P_{f,j}-A_j)=(C^{-1/2}P_{f,j}-C^{-1/2}A_j)^T(C^{-1/2}P_{f,j}-C^{-1/2}A_j)$

and in case $A_j\sim [\mu_{j}, C]$ and $P_{f,j}\sim [C^{-1/2}\mu_j, I_n]$ it holds that $C^{-1/2}A_j\sim [C^{-1/2}\mu_{j}, I_n]$ and $C^{-1/2}P_{f,j}\sim [C^{-1/2}\mu_{f,j}, I_n]$

It is not clear to me at all why the "Pseudo-MD" as defined above should be a sensical measure for the quality of prediction if the assumption of equal Covariancce-Matrix breaks down, as only one vector, namely the one on whose basis $\hat{C}$ was calculated (as defined in line 7) is normalised, why it is unclear what the multiplication with $C^{-1/2}$ does with the other RVs.

Furthermore it could be the case that it does not even hold that: $Cov(A_j)=Cov(A_i)$ then not even the RV $A_j$ would be normalised if we would estimate the covariance matrix as defined in line 7. Again the above defiend "Pseudo MD" would be nonsensical

So my question is basically whether my line of argumentation is correct or whether I have some misunderstandings about the way the MD works.

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ It sounds like you might want to read the replies at stats.stackexchange.com/questions/62092. $\endgroup$
    – whuber ♦
    Jun 30, 2018 at 12:37
  • 1
    $\begingroup$ One possible answer can be found from stats.stackexchange.com/questions/296361/…. In short, when both covariance matrices are equal, the Bhattacharyya distance coincides with the Mahalanobis distance, so one could look at the Bhattacharyya distance as a generalization of the Mahalanobis distance! $\endgroup$
    – kjetil b halvorsen ♦
    Jun 30, 2018 at 20:04

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.