In Bayesian inference, but also in MLE, the conditional independence assumption (CIA) is a usual assumption.

Typically, it is used to simplify the likelihood of the model, and to allow the factorization of the joint conditional distribution:

$$p(\mathbf{y}|\mathbf{\theta}) = \prod_{n=1}^{N}p(y_n|\mathbf{\theta})$$

where, $\mathbf{y}$ is the vector ($\mathtt{Nx1}$) of observations (measurements) and $\mathbf{\theta}$ contains the model parameters.


  1. When CIA is violated, it means that new realizations of $y$ depend on past observations. And clearly, if we assume independence, we can't trust the results. The data-generating model is wrong. Is this reasoning correct?

  2. How to test whether this is a reasonable assumption to make? Is it simply by looking at the auto-correlation function for the observations?

  3. In BDA 3, by Gelman et al., there is an Example: Checking the assumption of independence in binomial trials at page 147. How does that generalize to multiple regressors, so that $p(\mathbf{y}|\mathbf{\theta})$ is actually $p(\mathbf{y}|\mathbf{\theta}, \mathbf{X})$? and when $\mathbf{y}$ is a continuous variable?

  4. If there is high dependence (e.g. in time series), how can I get rid of it? One idea which comes to mind is to treat the problem as time-dependent, and model somehow the dependence.

  5. Do I understand it right that state-space models disguise this CIA under the Markovian process model construction?

    • If in reality $y$'s a are dependent (as it is usually the case in time-series), is it right to say that the dependence is "outsourced" to the hidden state?

If there are some other CIA-related aspects, which one should be aware of, and often overlooked, please share.

Thank you!



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.