2
$\begingroup$

I am trying to calculate mutual information on two observed continuous variables X and Y, which I believe to be dependent.

The formula relies on p(x,y): the joint probability density function of X and Y

Can I calculate p(x,y) from the observed values alone? (without knowing the real population distributions, or real dependency amount)

Can I use this general wikipedia formula: joint PDF

Or the dependecies between X and Y require a different approach?

I am a beginner in stats so please try to be not too technical. An answer with R code would be awesome!

$\endgroup$
2
$\begingroup$

The general wikipedia formula works, and no different approach is required to account for the dependencies between the random variables. The dependencies are already accounted for in the joint density function. The problem is in getting a good estimate of the joint density function if all you have are observed values $(x_i, y_i)$, $i = 1,2,\ldots, N$ especially if you only have a small sample in the case when your model is that $X$ and $Y$ are continuous random variables.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.