Timeline for Feature Selection: Information Gain VS Mutual Information
Current License: CC BY-SA 3.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 13, 2017 at 12:44 | history | edited | CommunityBot |
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
|
|
| Sep 26, 2014 at 19:20 | comment | added | Oleg Shirokikh | Thanks, Simone - it's resolved. Please see update in the question | |
| Sep 26, 2014 at 19:15 | vote | accept | Oleg Shirokikh | ||
| Sep 25, 2014 at 1:46 | comment | added | Simone | I just figured out that in these references the authors do not use Mutual Information but actually something called pointwise mutual information: en.wikipedia.org/wiki/Pointwise_mutual_information. Maybe try to post your examples here and the correspondent Weka outcome. | |
| Sep 25, 2014 at 0:58 | comment | added | Oleg Shirokikh | Yes, all categorical, clean. I'm just taking some toy examples, so I can verify the results by hand as well - and they do not match with Weka.. Also, I was able to find couple source (not sure how respectful they are) but they mention IG and MI as different measures: slideshare.net/guo_dong/feature-selection doras.dcu.ie/16194/2/… link.springer.com/chapter/10.1007/978-3-642-29075-6_11 jmlr.org/proceedings/papers/v10/sanasam10a/sanasam10a.pdf | |
| Sep 25, 2014 at 0:32 | comment | added | Simone | What are you comparing weka with? are all features categorical with no missing values? | |
| Sep 25, 2014 at 0:28 | comment | added | Oleg Shirokikh | That is good to know! It seems that it can be concluded that it is just a matter of ambiguous naming. The only open question is how Weka produces different results | |
| Sep 24, 2014 at 23:49 | comment | added | Simone | I guess the discrepancy originated from the first articles on decision trees: about ID3 and C4.5. It was appealing to maximize the information gain to the class due to a split according to a feature. Then this term started to be commonly used in machine learning. However this is just my guess. | |
| Sep 24, 2014 at 23:37 | comment | added | Oleg Shirokikh | Thanks, that's what I'm thinking this way too - as I wrote, math tells exact equivalence. Are you familiar with Weka? I'm wondering where is the discrepancy coming from. | |
| Sep 24, 2014 at 23:21 | history | answered | Simone | CC BY-SA 3.0 |