# What is “class” in mutual information based feature selection?

I'm having a little hard time understanding this specific feature selection algorithm.

Specifically, I am looking into maximum-relevance-minimum-redundancy method for feature selection.

If I have a feature matrix $\ X = \{x_1, x_2, ... , x_n\} , x_i \in R^D$

then I can compute mutual information between two specific individual features, but when talking about

relevance of that specific feature to a "class", in terms of mRMR algorithm,

what is exactly the definition of "class" ?

• If you look at Tom Cover's Information Theory book the result will be clear. – Michael R. Chernick Mar 8 '17 at 0:40

## 1 Answer

With no knowledge about this particular algorithm, I would guess that the word "class" is being used in the usual way in machine learning, which means "level of a discrete dependent variable". For example, if you're trying to build a classifier that distinguishes men and women, then there are two classes, men and women.

• But the mRMR algorithm requires that I find the mutual information of a feature to that "class", and I'm not so sure how I could calculate that. but, as Michael said, I will take a look at Tom Cover's book about information theory – Kevvy Kim Mar 8 '17 at 21:50