I have got a problem about doing a classification. I have got around 50 datasets. Each of them has 15 features.

I am trying to use these features to classify the 50 datasets to either 'Good' or 'Bad'. The ground truth labels of the 50 datasets are available that a classical training and validation can be done.

As there are 15 features, the problem should be considered as a classification in high dimensions. My question is:

Should we always perform PCA before we run any generic classification algorithms? Such as LDA, KNN or SVM.

I got someone's opinion that:

"PCA chooses the directions in which the variables have the most spread, not the dimensions that have the most relative distances between clustered subclasses."

But for my understanding, in order to do a better classification, we need to find features that have large differences between two groups. For example, we can calculate the mean and standard deviation of a feature for 'Good' and 'Bad' separately, and we can see if there are large differences. If so, we choose this feature. Also, we need to find features that have got least correlation in between. If two features have a large positive correlation, we can just choose to use one of them. PCA is somehow picking up the dimension reduced features for us, given 15 features, it will give 2 or 3 principle components that can be classified better. Am I right? Or I am on the wrong course?

Thanks for your suggestions.

A.

I have got a problem about doing a classification. I have got around 50 datasets. Each of them has 15 features.

I am trying to use these features to classify the 50 datasets to either 'Good' or 'Bad'. The ground truth labels of the 50 datasets are available so that a classical training and validation can be done.

As there are 15 features, the problem should be considered as a classification in high dimensions. My question is:

Should we always perform PCA before we run any generic classification algorithms, such as LDA, KNN or SVM?

I got someone's opinion that:

"PCA chooses the directions in which the variables have the most spread, not the dimensions that have the most relative distances between clustered subclasses."

But for my understanding, in order to do a better classification, we need to find features that have large differences between two groups. For example, we can calculate the mean and standard deviation of a feature for 'Good' and 'Bad' separately, and we can see if there are large differences. If so, we choose this feature. Also, we need to find features that have got least correlation in between. If two features have a large positive correlation, we can just choose to use one of them. PCA is somehow picking up the dimension reduced features for us, given 15 features, it will give 2 or 3 principal components that can be classified better. Am I right? Or I am on the wrong course?