We’re rewarding the question askers & reputations are being recalculated! Read more.

# Questions tagged [information]

The tag has no usage guidance.

42 questions
Filter by
Sorted by
Tagged with
5 views

### Analyzing 1000s of time-stamped tweets. How to automatically identify spikes in terms?

I have about 30,000 tweets from a corporate customer service account, all harvested according to the Twitter TOC. Naturally, each message is time-stamped and not extremely long. I'm trying to ...
15k views

### Measuring entropy/ information/ patterns of a 2d binary matrix

I want to measure the entropy/ information density/ pattern-likeness of a two-dimensional binary matrix. Let me show some pictures for clarification: This display should have a rather high entropy: ...
25 views

### Compute the information matries related to normal distribution

This is a problem that I have trouble with. Suppose that we have $X_{1}, \ldots, X_{m}$ are iid $N\left(\mu, \sigma^{2}\right), Y_{1}, \ldots, Y_{n}$ are iid $N\left(0, \sigma^{2}\right),$ the $X$ 's ...
27 views

### Nested sampling: estimate of bulk posterior support over prior

Going through the details of the Nested sampling Skilling paper, and I've encountered an estimate in Section 5 which I cannot reproduce. Rephrasing what's mentioned in the paper: we assume to have a ...
36 views

### Entropy evolution while learning?

It is fairly well known that $$H(X|Y)\le H(X),$$ the posterior entropy is smaller than the prior entropy. This is similar to $$\mathbb{E}_Y[\mathbb{V}ar_X[X|Y]]\le \mathbb{V}ar_X[X]$$ which follows ...
42 views

### Combining categories by Weight of Evidence

When calculating Information Value and Weight of Evidence, it's possible to draw a chart of WoE for each variable to study its effect on the state of the target variable. Now, I know it's possible to ...
202 views

### Fisher information matrix of two parameter exponential distribution

Is it possible to find the Fisher Information matrix of the two parameter (scale and location) exponential distribution? Any hint?
5k views

### Calculating the transfer entropy in R

The transfer entropy, from information theory, is an effective way to measure the one-way information dependence between two variables. A nice high-level summary is here: http://lizier.me/joseph/...
17 views

### Information Gain property

Studying about information gain I found in the web (from the presentation of a lecture) that $IG(C|X) = IG(X|C)$ is it true? How I prove it?
125 views

### Why not normalizing mutual information with harmonic mean of entropies?

This is a similar question to this one (which has unfortunately no answer yet), although I believe my question is more specific. Let $X$ and $Y$ be two discrete random variables with outcome space, ...
75 views

### Information about parameters using priors distributions [duplicate]

When using the "non-informative" prior $\pi(\mu,\sigma)\propto\frac{1}{\sigma^2}$ where $\pi(\mu)\propto1$ and $\pi(\sigma^2)\propto\frac{1}{\sigma^2}$ Where is the no information for the ...
276 views

### Bayesian Statistics -Prior and Posterior distributions

Please is it ever possible for the prior distribution to contain more information about parameter(s) than the posterior distribution? If yes, when can that occur? Is it the same concept as the ...
33 views

### Are there any measures of Nonspecificity that take into account unreachable states?

The Hartley function, the main measure of Nonspecificity, is essentially based on the count of all possible states. However, while coding the Hartley, I realized that "possible" means all remaining ...
22 views

### What are most recent research work on the problem of key phrases extraction from a text corpus?

I am interested in the problem of extracting key phrases from a text corpus. This is different from the keyword extraction problem, which is only for a particular document. This problem helps us, for ...