I just read these two quotes in a paper and wondered how I do this.

For each classifier, one selects a subset of the input features according to their correlation to the corresponding class.


Compute the correlation between each feature and Yi the output for class i.

Source: Dimensionality Reduction Through Classifier Ensembles

Can you tell me how to accomplish that?


2 Answers 2


Correlation is used as a method for feature selection and is usually calculated between a feature and the output class (filter methods for feature selection). It roughly translates to how much will the change be reflected on the output class for a small change in the current feature. If the change is proportional and very high, then we say that the feature is highly correlated with the output class and is usually a very good idea to keep it around for any end of the pipeline tasks.

That being said, a feature having higher correlation in say corpus XYZ won't necessarily mean that it will have a high correlation in some other corpus say ABC and thus cannot be translated from one dataset to another.

There are many metrics to measure the correlation between a feature and a class label such as mutual information, chi-square, correlation coefficient scores etc.

numpy.correlate in python does the trick for most of my work. or if i want to use mutual information then sklearn has mutual info score .


This is called correlation filter. If you are using R, You can have a look at FSelector package's linear.correlation and rank.correlation functions. When class labels are categorical, you can use Point-Biserial Correlation, polyserial or polychloric correlation.

The idea is if feature 1 is highly correlated with class label while feature 2 is not highly correlated we would select feature 1 over feature 2.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.