2
$\begingroup$

I was reading on page 245 of Hastie et al. (Elements of statistical learning) about how not to do feature selection (basically they describe what happens when independence between test and training set is not given due to using the entire data for feature selection). There they perform the following experiment to exemplify their point:

"Consider a scenario with N = 50 samples in two equal-sized classes, and p = 5000 quantitative predictors (standard Gaussian) that are independent of the class labels. The true (test) error rate of any classifier is 50%. We carried out the above recipe, choosing in step (1) the 100 predictors having highest correlation with the class labels, and then using a 1-nearest neighbor classifier, based on just these 100 predictors, in step (2). Over 50 simulations from this setting, the average CV error rate was 3%. This is far lower than the true error rate of 50%."

I just have a detail question: This is a two-class problem, so I assume the labels are for instance either 1 or 0 - how is it possible then to calculate correlation values between features and labels? I understand that this can be done in the regression context where the response variable can take on any value, but I am not sure about the classification context. Thanks

$\endgroup$
1
$\begingroup$

You will do it exactly in a same way as with continuous variables. Only your dependent variables have value of 0 or 1. You Pearson's correlation will basically became a t-test

$\endgroup$
  • $\begingroup$ ok so assuming that my features are real values between say 0 and 10, and I have 10 data samples in total (5 for each class), I would for instance calculate for one particular feature the following: corr( [0.3,6,0.9,8,7,0.5,2.2,5.4,0.1,9] ,[1,0,1,0,0,1,1,0,1,0]) - in how far is that like a t-test? $\endgroup$ – user24544 Nov 11 '15 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy