LDA tries to maximise the ratio of between-class-scatter to within-class-scatter. That is - it seeks to find a projection where there is a big gap between the classes and small variance within one class.
The problem with scenarios where there are more features than samples is that there often is an infinite number of projections where within-class-scatter is 0. Hence there are infinite number of solutions with seemingly infinite gap between the classes.
A typical way to deal with this problem is to regularize the within-class-scatter matrix. And there are many different ways to go about it.
The simplest way to do this is by adding additional variance in all directions. To achieve this: add a small constant number to all the diagonal elements of the within-class-scatter matrix.
In order to use principal component analysis in combination with LDA you would do the following:
- Compute the principal axes of PCA using your "training set" of faces.
- Determine the number ($n$) of principal components you want to retain.
- Project both the training and testing sets using the chosen number ($n$) of axes estimated from the training set. This will reduce the feature size of your samples to ($n$)
- Apply LDA on top of the projected samples.
A closely related solution is using the Moore–Penrose pseudoinverse instead of inverse of the within-class-scatter matrix. And there are many more different tricks and types of regularizations that can be applied, depending on context and assumptions.