The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [mutual-information]

mutual information is a concept from information theory. It is a measure of joint dependence between two random variables, which is not, like the usual correlation coefficient, limited to scalar variables.

239 questions
Filter by
Sorted by
Tagged with
14 views

measure of the difference between variation of two time series of probability distributions

I'm looking at a series of particle density probabilities of proteins floating on a cell, these particles move around and also blink, so these density probabilities differ a bit from one time to ...
9 views

71 views

Cross - entropy for two variables with different prob. distributions

Let us say that we have given two random variables with different prob. distributions: A = [0.1, 0, 0.5, ...] B = [0.3, 0.1, 0.03, ...] What should I do when I want to compute the reformulated cross-...
50 views

Is mutual information symmetric under conditioning?

Cover defines conditional mutual information as I(X; Y|Z) = H(X|Z) - H(X|Y,Z), which I found confusing, as I would have expected I(X; Y|Z) = H(X) - H(X|Y,Z). So I started trying to see which of 1) I(...
69 views

Relative Entropy decomposition

Can the relative entropy (Kullback Leibler divergence) between multivariate distributions be decomposed into relative entropies of the different variables plus some measure of dependence between the ...
72 views

Infinitesimal independence

Let's say we have two random variables $X$ and $Y$. Is there a name for saying that $X$ and $Y$ are independent only for the values concentrated around a small interval around some $x_0$ and $y_0$ ? ...
358 views

Which result for normalized mutual information is correct?

I wanted to find the normalized mutual information to validate a clustering algorithm, but I've encountered two different values depending on the library I use. In Python: ...
59 views

Does downsampling decrease the entropy of the data?

Suppose we have an $n-dim$ time-series $X={x_1, x_2, \cdots, x_n}$ and we resample it to $m-dim$, $\hat{X}={\hat{x}_1, \hat{x}_2, \cdots, \hat{x}_m}$, where $m < n$. Can we say this downsampling ...
The question Let $X \sim \mathcal{N}(0,1)$ be a random variable denoting the location of a target on the real line. $Y_a$ be a binary random variable encoding the (noisy) answer to the question: "is ...