# Correct understanding of De Finettis representation theorem

I am currently interestend in understanding De Finettis representation theorem. As I am only familiar with Frequentist thinking I have some problems to understand its meaning. I have already read the great answer by Zen at: What is so cool about de Finetti's representation theorem?, but there are still a couple things that I don't understand.

Recall that De Finetti's Representation Theorem says that $\{{X_i}\}_{i=1}^\infty$ is exchangeable if and only if there is a random variable $\Theta:\Omega\to [0,1]$, with distribution $\mu_\theta$, such that

$$p(X_1=x_1, ..., X_n=x_n)=\int_{[0,1]}\theta^{\sum_{i=1}^n{x_i}}(1-\theta)^{n-\sum_{i=1}^n{x_i}}d\mu_\Theta$$

Furthermore De Finettis strong law of lare numbers says that:

$$\overline{X}_n=\frac{1}{n}\sum_{i=1}^nX_i\to\Theta \quad a.s. \text{ for }n \to \infty$$

1. How how am I to interprete this random variable $\Theta$ and its realisation $\theta_T=\Theta(\omega)$. Can I think of this parameter as being randomly drawn once from a box of tickets, therefore being constant but unknown to the observer?

2. Why isn't this parameter always the same (or is it the same, e.g. in the case of repeated coin tosses)? Can you give an example or an intuitive explanation that clarifies this idea? Or framed differently what does $\mu_\Theta$ stand for? Is this Parameter just a construction in our head which - if we knew it - would enable us to construct the beliefs we would have, if we had seen all the outcomes?

What is the proper interpretation of the parameter? The natural interpretation of the parameter $\Theta$ comes from the law-of-large numbers, which you have stated in your question. This says that the following equivalence holds almost surely:

$$\Theta = \lim_{n \rightarrow \infty} \frac{1}{n} \sum_{i=1}^n X_i.$$

From this equation we see that $\Theta$ is the limiting proportion of positive outcomes ($X_i=1$) in the observeable sequence.$^\dagger$ Since you are familiar with frequentist thinking, you will recognise this as the frequentist definition of the probability of a positive outcome in the sequence. In fact, there is a good argument to be made that de Finetti's theorem is a statement of the equivalence that is used in frequentist theory as the definition of probability.

To give an applied example of this interpretation, suppose that the outcomes $X_i$ are indicators for a sequence of coin flips, indicating an outcome of heads. In this case the observable sequence $X_1,X_2,X_3,...$ is an exchangeable sequence of coin flips and the parameter $\Theta$ is the long-run limiting proportion of heads in the sequence.

In your question you also ask if the parameter is "just a construction in our heads". Well, it is no more of a construction than the notion that there is an infinite sequence of outcomes in the first place. If we are willing to assume that there is an infinite sequence of observable outcomes (which is itself arguably a mere hypothetical construction) then that sequence has a limiting proportion of positive outcomes$^\dagger$, and that limiting proportion is the parameter.

$^\dagger$ There is a slight technicality here, since the limiting proportion is a Cesaro limit that does not exist for all possible sequences. The parameter can be defined as the Banach limit, which always exists; for discussion, see O'Neill (2009).

Is the parameter an unknown constant? In classical frequentist analysis, the parameters are treated as "unknown constants". There is no such thing as an unknown constant in Bayesian statistics, since all unknown quantities are treated as random variables with a prior distribution. Hence, if you are going to treat $\Theta$ as an unknown constant, you are using frequentist analysis, not Bayesian analysis.

Why isn't the parameter always the same? As shown above, the parameter $\Theta$ represents the limiting proportion of positive outcomes in the sequence. Clearly, one can imagine that there are different sequences of exchangeable outcomes where the limiting proportions of positive outcomes are different, and hence, the parameter would not be the same across different sequences. For example, a particular exchangeable sequence of coin-tosses might have a limiting proportion of heads of $\Theta = 0.5$, meaning that we are dealing with a "fair" coin. Alternatively, we might have an exchangeable sequence of coin-tosses with a limiting proportion of heads of $\Theta = 0.51$, meaning that we are dealing with coin that is "biased" towards heads.

What does $\mu_\Theta$ stand for? This is the prior probability measure for the parameter (equivalent to its prior distribution). In Bayesian analysis, this measure represents our beliefs about the unknown parameter prior to seeing the data. After we see the data we then update this to a posterior belief, represented by the posterior probability measure (equivalently a posterior distribution).

• Thanks for the great answer. There is one more question left: According to which probability measure does the equivalence hold almost surely? I guess according to the prior $\mu_\theta$? – Sebastian Sep 17 '18 at 16:42
• @0rangetree: That would be the overall probability measure on $\Omega$ (i.e., with probability one, the Cesaro limit of the sequence $X_1,X_2,X_3,...$ will be equal to the parameter $\theta$. If you have a look at the paper linked in my question you will see a useful discussion of this aspect of the theorem. – Ben Sep 17 '18 at 23:14
• @0rangetree: It appears that you have now asked this as a new question (a good idea). I have added a more complete answer there. – Ben Sep 18 '18 at 3:15