3
$\begingroup$

De Finetti's theorem states that, if observations $(x_1, x_2, x_3, \cdots)$ are infinitely exchangeable, then their joint probability $p(x_1, x_2, \cdots, x_N)$ has a representation as a mixture:

$$p(x_1, x_2, \cdots, x_N) = \int(\Pi_{i=1}^N p(x_i | \theta)) dP(\theta)$$

for some random variable $\theta$ (the definition is from a lecture by Michael Jordan).

If this is true and if our data is sampled iid (which implies exchangeability), then why aren't we using this theorem for things like linear regression and logistic regression?

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ It would perhaps be more directly relevant in the context of something like a permutation test -- which is sometimes used with linear regression. $\endgroup$ – Glen_b♦ Oct 16 '16 at 2:18
  • 4
    $\begingroup$ How would you suggest we may use this theorem for regression? $\endgroup$ – Richard Hardy Oct 16 '16 at 9:57
  • 1
    $\begingroup$ In fact, in the iid case this holds for any $\theta$ independent of $X$, e.g. $\theta = 0$. So it's pretty unclear how this would be useful in standard regression setups. $\endgroup$ – Dougal Oct 16 '16 at 13:06
1
$\begingroup$

De Finetti's representation theorem is generally taken to be a justification for the assumption of IID observations conditional on underlying parameters. The latter is a model specification that jumps straight to assumptions about unobservables, whereas the former is a purely operational assumption applying to observables. It is therefore common to assume infinite exchangeability as a means to get to the IID model.

Within Bayesian regression this is the standard way you would frame your analysis. You would begin by assuming that the sequence of observables is exchangeable, which gives you the IID form, and then you would go further and assume that the long-run behaviour of the observables conforms to some specified regression model (which assumptions you would then test later).

|cite|improve this answer
$\endgroup$
1
$\begingroup$

It is used extensively, but it is still a tool in a giant set of tools. If you have prior knowledge of the parameters or coherence is a key element of the work you are doing, then it should clearly be used.

If, on the other hand, you find a need for a guaranteed minimum rate against false positives or controlling power is important, then you would use a Pearson and Neyman method. Likewise, if your goal is epistemological and you need to rule out a wide class of alternatives, Fisher's method may be preferable. And, finally, if you do not know the functional form, you may consider neural networks.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.