2
$\begingroup$

In Bayesian inference, but also in MLE, the conditional independence assumption (CIA) is a usual assumption.

Typically, it is used to simplify the likelihood of the model, and to allow the factorization of the joint conditional distribution:

$$p(\mathbf{y}|\mathbf{\theta}) = \prod_{n=1}^{N}p(y_n|\mathbf{\theta})$$

where, $\mathbf{y}$ is the vector ($\mathtt{Nx1}$) of observations (measurements) and $\mathbf{\theta}$ contains the model parameters.

Questions:

  1. When CIA is violated, it means that new realizations of $y$ depend on past observations. And clearly, if we assume independence, we can't trust the results. The data-generating model is wrong. Is this reasoning correct?

  2. How to test whether this is a reasonable assumption to make? Is it simply by looking at the auto-correlation function for the observations?

  3. In BDA 3, by Gelman et al., there is an Example: Checking the assumption of independence in binomial trials at page 147. How does that generalize to multiple regressors, so that $p(\mathbf{y}|\mathbf{\theta})$ is actually $p(\mathbf{y}|\mathbf{\theta}, \mathbf{X})$? and when $\mathbf{y}$ is a continuous variable?

  4. If there is high dependence (e.g. in time series), how can I get rid of it? One idea which comes to mind is to treat the problem as time-dependent, and model somehow the dependence.

  5. Do I understand it right that state-space models disguise this CIA under the Markovian process model construction?

    • If in reality $y$'s a are dependent (as it is usually the case in time-series), is it right to say that the dependence is "outsourced" to the hidden state?

If there are some other CIA-related aspects, which one should be aware of, and often overlooked, please share.

Thank you!

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.