I'm having a little hard time understanding this specific feature selection algorithm.

Specifically, I am looking into maximum-relevance-minimum-redundancy method for feature selection.

If I have a feature matrix $\ X = \{x_1, x_2, ... , x_n\} , x_i \in R^D $

then I can compute mutual information between two specific individual features, but when talking about

relevance of that specific feature to a "class", in terms of mRMR algorithm,

what is exactly the definition of "class" ?

  • 1
    $\begingroup$ If you look at Tom Cover's Information Theory book the result will be clear. $\endgroup$ Mar 8, 2017 at 0:40

1 Answer 1


With no knowledge about this particular algorithm, I would guess that the word "class" is being used in the usual way in machine learning, which means "level of a discrete dependent variable". For example, if you're trying to build a classifier that distinguishes men and women, then there are two classes, men and women.

  • $\begingroup$ But the mRMR algorithm requires that I find the mutual information of a feature to that "class", and I'm not so sure how I could calculate that. but, as Michael said, I will take a look at Tom Cover's book about information theory $\endgroup$
    – Kevvy Kim
    Mar 8, 2017 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.