2
$\begingroup$

In Multivariate Analysis (Mardia et al, pg 324), the authors write

"it might be thought that a linear combination of two variables would provide a better discriminator if they were correlated than if they were uncorrelated".

Before studying the problem, why would one hypothesize this? My guess (which was incorrect) is that uncorrelated variables may discriminate better because they are not redundant.

My question is not about the proof against statement above. Rather: why does Mardia propose that the readers may have expected what is written above when I am guessing the opposite?

$\endgroup$
2
$\begingroup$

I don't have the book at hand at present, so I can't be sure what the authors meant and in what context. I will surmise that they are speaking of the most straightforward evidence. And the key is that they - I suppose - talk about some single linear combination, that is, single variable derived from two variables. "a linear combination of two variables would provide a better discriminator if they were correlated than if they were uncorrelated".

enter image description here

Correlated variables like on the left mean that the classes are aligned roughly along one line. It is this line that pretends to be the single discriminant (discriminator). As the cloud approaches straight sausage in shape the class centroids approach the red line. SSbetween for that one discriminant becomes maximal and almost exhausts the overal 2D dispersion of the centroids.

If the cloud is about round like a concretion of classes - the right pic - so that no correlation between the variables occur, no single line could pretend to be the sole discriminator. A red discriminant 1 must be supplemented by a green discriminant 2 in order to explain the full scatter of the centroids. Moreover, because the configuration is round, a discriminant's direction in space is not so uniquely determined as on the left: if we rotate it a bit it won't lose much of its as it is weak discerning power.

The key point is the shape of the total cloud rather than correlation per se. Below there is no correlation but the sausage persists, and the discrimination by a single discriminant is as good as on the first pic. On the first pic, both V1 and V2 variables contribute about equally to the linear combination which the discriminant is; on the last pic, only V1 contributes to it much, actually - coincides with it.

enter image description here

The issue we were speaking is the issue how effective is discriminant analysis as the dimensionality reducer for this or that dataset.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.