I understand KL divergence abstractly, but I'm not exactly sure how you would calculate it for a multivariate binomial distribution (such as an Ising model on a random graph). If I am sampling 100 binary variables that co-vary, what would the equation look like? I know the equation for multivariate normal distributions, but if I use that equation, I get negative KI.
If someone could point me in the right direction I'd greatly appreciate it, I'm a biophysicist just moving into information theory, so I'm while I understand operations on probability distributions in theory, I'm not always sure how to apply them practically. Thanks!
