# Why can we assume independence Bayesian inference?

Scenario: A series of N points(${x_{1,..,n}}$) are drawn from distribution p(x|$\theta$),here $\theta$ is a parameter for distribution and itself being a random variable. After getting evidence from N data points, we get a better(less variability) distribution for $\theta$,in short we are guessing value of $\theta$ using Bayes theorem $$P(\theta \mid {x_{1,..,n}}) = \frac{P(x_{1,..,n} \mid \theta) \, P(\theta)}{P(x_{1,..,n})}$$

I get from other literature that we calculate $P(x_{1,..,n} \mid \theta)$ as $\prod_{k=1}^n P(x_{k} \mid \theta)$ ,this assumes that observations are independent

How well is the assumption justified? Since each observation adds some information to guess $\theta$,next observation totally depends on previous one, in fact exploiting this evidence clues we even try to predict what'd be distribution for next observation by marginalising posterior over prior

$$P(x_{new} \mid x_{1,..,n}) =\int_\theta P(x \mid \theta) P(\theta \mid {x_{1,..,n}}) d\theta$$

• I edited your title to something more meaningful, feel free to change it if you do not find it appropriate.
– Tim
Commented Oct 29, 2017 at 14:32

We only assume that the $x_i$'s are independent conditional on $\theta$, that is, $$P(x_{1,..,n} \mid \theta)=\prod_{k=1}^n P(x_{k} \mid \theta).$$ This means that if we think of the model parameters as some fixed constants, then the $x_i$'s are independent which is often reasonable. The model parameters $\theta$ are fixed constants but we still use a probability distribution to represent our limited state of knowledge about them.

Even though we have conditional independence, this do not imply marginal independence, however, that is, when integrating out $\theta$, we no longer have $$P(x_{1,..,n})=\prod_{k=1}^n P(x_{k})$$ and the distribution of $x_{new}$ conditional on the data (the posterior predictive density) indeed depend $x_{1,..n}$.

• While the answer is correct, it might be a good idea to say what actually the conditional dependence is (it is possible that OP did not hear about it and this has led to confusion).
– Tim
Commented Oct 29, 2017 at 14:26
• Yes Tim I didn't understand answer :(
– user182728
Commented Oct 29, 2017 at 14:30
• @ZaidSyedMMd I added some more explanation. Commented Oct 29, 2017 at 16:16
• @ZaidSyedMMd if it does please consider "accepting" the answer (the "v" button under the score on the left).
– Tim
Commented Oct 31, 2017 at 9:46