When is the scatter matrix in linear discriminant analysis singular?

In linear discriminant analysis (LDA), when there are fewer data instances than the number of dimensions (i.e., when the data matrix is of order $n \times m$ where $n$ is less than $m$), the within-class scatter matrix $S_W$ is said to be singular. Does this mean that a scatter matrix generated from any $n\times m$ data matrix where $n$ is less than $m$ is guaranteed to give a singular $S_W$ matrix? How much smaller does $n$ have to be in comparison to $m$ for $S_W$ matrix to be guaranteed singular?

• Yes, it means exactly that. Sep 1, 2015 at 23:08