New answers tagged

1 vote

Measure the Mutual Information between two variables that is also shared by a third variable

I know of two points that might be relevant to your question: First, there is the data processing inequality, which, for a Markov chain $X\to Y\to Z$ states: $$ I(X, Y) \ge I(X, Z). $$ This gives you ...
user avatar
  • 6,463
0 votes

Mutual information relationship to copula entropy is borked?

I'll try to provide some intuition: As you already mentioned, differential entropy may become negative and that is exactly what happens here. Recap that differential entropy is some measure for the ...
user avatar
  • 11
0 votes
Accepted

Let $\textbf{y}=AX\textbf{w}+\textbf{z}$ where $\textbf{w}$ and $\textbf{z}$ are i.i.d. vectors, why does $p(\textbf{z})=p(\textbf{y}|X,\textbf{w})$?

Your derivation for the formula of $p(\mathbf y|\mathbf w)$ is correct (I will often drop the parameters $A$ and $X$ from the formulae). Next, a minor point: note that in the formula for $\int p(\...
user avatar
  • 6,463
1 vote

mutual information between discrete random variables and continuous variables

Note that there is a difference between defining and calculating the MI of two random variables $X$, $Y$ whose joint distribution you know and actually estimating the MI from data $\left(X_i, Y_i\...
user avatar
  • 518

Top 50 recent answers are included