# How to correctly apply LDA following PCA?

I have a high dimensional dataset ($n \times p$: $30 \times 100$) which I want to use as an testing dataset to build a two group classifier (LDA or QDA). I've read that you can do PCA to do an dimension reduction of your dataset to select to most important features. But I'm a bit confused what you use exactly as the input to build the classifier. I'm familiar with PCA using SVD and what it means.

Consider following situation:

• I do a SVD of my dataset.
• I look at the scores of the first couple of principal components.
• When I assign my scores a label indicating from which group they come, I see that the 3th PC best separates my 2 groups (although it only explains 7% of the total variance).

What do I do next?

1. I take the 3th PC transform to the original parameter space (scores * loadings * scale + mean) and build my classifier
2. I look at the loadings in the 3th PC and try to decide which parameters in my original parameter space are important and build a classifier only using these.
3. ...

Option 2 seems the most sensible in my opinion but I'm not entirely sure. Also If I see that only the 3th PC is important to explain the variance in my two groups, can I forget about the first two PC in my further analysis?

• It is unclear whether you want just dimensionality reduction (given that you have n<p) or you want feature selection (i.e. you insist to stay only with very few variables out of 100 before you proceed to LDA). – ttnphns May 14 '14 at 20:12
• If your wish is the first one I said about, i.e. only the fact that n<p bothers you and only this fact forces you to apply PCA, - then you should retain 28 (all but one last) components in PCA and do LDA on those 28 components, leaving there few strongest discrimimants. Each of those is the linear combination of 28 components, each of which is in turn the linear combination of 100 variables. – ttnphns May 14 '14 at 20:33
• – amoeba says Reinstate Monica Dec 22 '14 at 15:41