# Questions tagged [information]

The tag has no usage guidance.

39 questions
Filter by
Sorted by
Tagged with
0answers
29 views

### Entropy evolution while learning?

It is fairly well known that $$H(X|Y)\le H(X),$$ the posterior entropy is smaller than the prior entropy. This is similar to $$\mathbb{E}_Y[\mathbb{V}ar_X[X|Y]]\le \mathbb{V}ar_X[X]$$ which follows ...
0answers
23 views

### Combining categories by Weight of Evidence

When calculating Information Value and Weight of Evidence, it's possible to draw a chart of WoE for each variable to study its effect on the state of the target variable. Now, I know it's possible to ...
0answers
17 views

### Information Gain property

Studying about information gain I found in the web (from the presentation of a lecture) that $IG(C|X) = IG(X|C)$ is it true? How I prove it?
1answer
119 views

### Why not normalizing mutual information with harmonic mean of entropies?

This is a similar question to this one (which has unfortunately no answer yet), although I believe my question is more specific. Let $X$ and $Y$ be two discrete random variables with outcome space, ...
0answers
74 views

### Information about parameters using priors distributions [duplicate]

When using the "non-informative" prior $\pi(\mu,\sigma)\propto\frac{1}{\sigma^2}$ where $\pi(\mu)\propto1$ and $\pi(\sigma^2)\propto\frac{1}{\sigma^2}$ Where is the no information for the ...
2answers
194 views

### Bayesian Statistics -Prior and Posterior distributions

Please is it ever possible for the prior distribution to contain more information about parameter(s) than the posterior distribution? If yes, when can that occur? Is it the same concept as the ...
0answers
22 views

### What are most recent research work on the problem of key phrases extraction from a text corpus?

I am interested in the problem of extracting key phrases from a text corpus. This is different from the keyword extraction problem, which is only for a particular document. This problem helps us, for ...
1answer
28 views

### Finding Fisher Matrix for Line Fitting

I am going through the "Fitting a Line" example from here. $f_1 = ax_1 + b$ and $f_2 = ax_2 + b$ are the models used to observed two data points in $R^2$. If $\sigma_1$ and $\sigma_2$ is the ...
0answers
72 views

### Mutual Information from Multiple Sources

The mutual information gain expression is $$H(X) - H(X | Y)$$ If I have a set of data sources, $\mathbf{X} = \lbrace X_0, X_1,\ldots,X_m \rbrace$, then I start with the simplest mutual ...
0answers
116 views

### What is information gain ratio?

With respect to data mining, what is information gain ratio? I'm a complete beginner to data analytics and mining, so please explain at a low level of understanding.
0answers
26 views

### Identifying / visualising the most informative data source or combination of data sources

I have observations on the status (say, dead = 0, alive = 1) of a number of subjects as recorded in three distinct data sources at the same point in time: +---------+---------+---------+---------+ | ...
0answers
76 views

### Difference between two entropic states of the same variable

I am interested in finding the difference of entropy between two states of the same variable. The probability distribution associated with the variable X changes at each discrete point in time t by an ...
1answer
58 views

### Metric for temporal deviation or variation from mean? Hurst or one-sample K-S test?

I am not sure I am articulating this question properly, and my unfamiliarity with the proper terminology has hindered my ability to research this question. I am looking for a metric that captures ...
0answers
226 views

0answers
61 views