Could someone explain to me the difference between within-group covariance matrix and between-group covariance matrix in the context of linear discriminant analysis?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ What exactly is it that you don't understand? $\endgroup$– A. DondaCommented Aug 23, 2015 at 16:48
-
$\begingroup$ The definitions are given in this question: stats.stackexchange.com/questions/8625. $\endgroup$– amoebaCommented Aug 24, 2015 at 19:06
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
Within-group covariance matrix is the average of covariance matrices of each group, weighted by the groups' weight. And between-group covariance matrix is the covariance matrix of the group means (centroids), weighted by the groups' weight.
What LDA aims to achieve, is minimal variance within groups and maximal variance between groups.
-
$\begingroup$ So does that mean if there are K classes of outcomes, there are K within group covariance, but only 1 between group covariance? $\endgroup$– JimCommented Aug 23, 2015 at 22:47
$\begingroup$
$\endgroup$
Imagine two classes: C1 and C2. In the context of LDA, the within-group (W) and between-group (B) covariances mean this:
W: Measures the variance of samples in each class, for example, the variance of samples in C1, or that of samples in C2.
B: Measures the variance of samples across different classes, for example, how far away is the sample mean of C1 to that of C2.
Why is Fisher interested in W and B in the context of LDA? Because he thinks the best way to process the data for classification is to find a projection direction on which the projected data have:
- Minimum within-class covariance
- Maximum between-class covariance.
To see why this rule actually makes sense, and more details on LDA, you can read my article on linear discriminant analysis.
