To put it in context, I was trying to learn Bayesian parameter estimation (by an example of learning the probability of heads of a coin) and was trying to understand the independence of the samples that we get, specifically, how marginal independence of each sample depends on the assumption of our model. Specifically, how unknown and fixed affects independence.
In the frequentist view, we consider a parameter $\theta$ that is is unknown but fixed. How and why does that allow us to consider that then our samples are independent? For me intuitively, that doesn't completely make sense because each coin toss tells us something about the parameter $\theta$ that we are trying to learn and hence, telling us about the probability of the next toss. Why does assuming that a parameter is fixed, removes the other uncertainty and somehow makes the samples independent?
To put my question even more in context, consider the extract of Probabilisitc Graphical Models (by Koller and Friedman) that lead to my confusion/misconception:
For completeness consider their fig 17.3:
The issue is not the graphical model. I've studied these before. I do understand that observing the model separates (D-separates in fact) the r.v's $X_i$, which are the samples. I do understand conditional independence, and that is not confusing me either. However, what is confusing me, is why we are allowed to assume independence in the frequentist view. How come the argument Koller and Friedman doesn't break down in the frequentist case. Specifically, the following sentence confuses me:
If we do not known $\theta$, then the tosses are not marginally independent: Each toss tells us something about the parameter $\theta$, and thereby about the probability of the next toss.
That makes sense to me in the graphical model. However, it seems that argument could be used in the frequentist case too, since the model is unknown. What is special about the fixed that makes this issue magically disappear?
I think this is more of a intuitive/conceptual issue rather than a mathematical misunderstanding, but I am not sure. Anyway know how "fixed" fixes the independence issue?