Questions tagged [information-gain]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Information Gain Ratio

I tried ranking some variables using WEKA cross-validated information gain ratio. Apart from five of the variables, the other variable average merit came back as 0. is that normal?
user avatar
1 vote
1 answer
28 views

Which method is best to find the correlation between 2 datasets in my case

I have multiple datasets that i need to find the correlation between them The problem is that my datasets are mainly zeros and ones (zero means patient does not have the disease and 1 means patient ...
user avatar
  • 249
0 votes
0 answers
29 views

Information Gain as a measure of term importance

I was watching the talk Beyond TF-IDF: Why, What and How. The speaker suggests replacing Inverse Document Frequency (IDF) with "Information Gain". Say we have a corpus of documents each ...
user avatar
  • 205
1 vote
2 answers
70 views

feature importance aggregation

I have more of a conceptual question I was hoping to get some feedback on. I am trying to run a boosted regression ML model to identify a subset of important predictors for some clinical condition. ...
user avatar
  • 107
1 vote
1 answer
70 views

The correct way to form a PDF from probabilities of means (to calculate Entropy later)

The Question I have a probabilistic belief. I have 4 normal distributions, and I believe in each one a different amount. As follows: $$ P(M=ceramic)=0.9987 \\ P(M=aluminium)=0.0013 \\P(M=plastic)=0 \\ ...
user avatar
1 vote
0 answers
22 views

How feature is selected to be a decision node after splitting the first root node using entropy in decisions trees?

In this article on how entropy is calculated and how the decision tree is built, the writer chose to split the sunny branch into ...
user avatar
  • 143
0 votes
0 answers
93 views

Who to attribute information gain to?

I am writing a paper where I examine information gain specifically with regards to feature selection and am wondering what the proper reference should be. I have looked all over and I can't find a ...
user avatar
  • 1,408
2 votes
1 answer
110 views

Continuous variable evaluation in decision trees

I was going through the C4.5 and ID3 algorithms used to construct a decision tree. Was wondering if there is an efficient way to compute information gain from a continuous variable (during the step ...
user avatar
  • 305