0
$\begingroup$

I am writing a paper where I examine information gain specifically with regards to feature selection and am wondering what the proper reference should be. I have looked all over and I can't find a good reference other than potentially referencing Shannon's original paper for entropy. It looks a little awkward in the paper since all of my other feature selection methods I mention I attribute to someone but I don't have one for information gain.

Edit: Put another way since apparently the above is hard to follow... I’m writing a paper, in my paper I use information gain and briefly describe it. I would like to know which paper to put in my references. For example if I talk about random forest I would say “we classify using random forest (Breiman, 2001)...” and then I would list Breiman’s paper in my references. Thing is (as I say in the question) I can’t find a paper where information gain is first introduced and so I don’t know what to cite and which author to attribute the method to.

|cite|improve this question
$\endgroup$
  • $\begingroup$ How can a reference be opinion based? That makes no sense, all I am asking for is if anyone knows the original paper that talks about information gain? There is no opinion involved... $\endgroup$ – astel Sep 19 at 14:52
  • 1
    $\begingroup$ Several of those voting to close invoked a different reason, saying that your question is unclear. I agree it doesn't appear to be opinion-based, but I am struggling to understand what you are looking for, for two reasons. First, it's not evident what you mean by "information gain" (this phrase can be used in many senses) and second, you point to a paper (Shannon's) that if relevant would obviously be the original, so why is there a need for further research into the literature? $\endgroup$ – whuber♦ Sep 19 at 15:00
  • $\begingroup$ To answer your questions: 1) I specifically say in my question I am using information gain with respect to feature selection. 2) I mention Shannon's paper as something to cite, but if you were familiar with information gain for feature selection you would know that entropy is only involved in the calculation of information gain and is not the actual method and as such is not really the proper attribution. Not understanding the method I am describing is not reason to close a question. $\endgroup$ – astel Sep 19 at 16:08
  • $\begingroup$ Please, then, elaborate on what you mean by "proper attribution." Not understanding what you're trying to ask is an excellent reason to close a question: it prevents misunderstandings when answers based on multiple different interpretations appear. $\endgroup$ – whuber♦ Sep 19 at 16:12
  • 1
    $\begingroup$ Putting together feature selection and information gain I think we go to classification with decision trees, therefore the farthest reference I know about is J. R. Quinlan - Induction of decision trees, or ID3. $\endgroup$ – rapaio Sep 20 at 5:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.