# Choosing the best featureset for prediction

I have this N sets of features F each with $F_i$ number of features. All the feature sets have 20000 examples and we have 20,000 labels for them.

Lets say feature set $F_1$ has 10 features and feature set $F_2$ has 20 features. Each of these feature sets have 20000 examples. Now all these examples have labels.

So I want to decide which of these feature sets is useful for the classification. I don't want to create a classifier and decide because my labels are highly unbalanced.

I was thinking of doing this with canonical correlation. I will take one feature set $F_i$ and the label vector and do cannonical correlation, get the score.

Do that for all the feature sets and then sort them based upon the scores obtained. How would that be?

First, get all of the $p$ features known, and all of the $n$ examples that are known, and construct an $n \times p$ data matrix $\mathbf{X}$. Then, answer the question: Which features predict class membership best using all the training data? This is then followed by breaking up the examples into folds to run CV, for which some groups claim you should identify the best features which best predict class membership of example in the fold left out of training. Other groups claim you can do what you are doing during CV feature identification.