2
$\begingroup$

I am reading the Tutorial of Fisher Information and it is mentioned that the say we have a random variable $X^n$ where $n$ refers to number of tosses so the RV $X^2$ could be {$1,1$} or {$0,1$} then the number of head given $\theta$ is given by the following binomial distribution formula, here $\theta$ determines coint is fair or biased:

$$p(y|\theta)=\begin{pmatrix}n\\y \end{pmatrix}\theta^y(1-\theta)^{n-y}\ , \textrm{where y is amount of heads}$$ $$ \begin{pmatrix}n\\y \end{pmatrix}=\dfrac{n!}{y!(n-y)!}$$

Then it mentions

Observe that the conditional probability of the raw data given Y = y is equal to $$P(X^n|Y=y,\theta)={1\over{\mathbin{}\begin{pmatrix}n\\y \end{pmatrix}}}$$

Now what I dont understand why is the above conditional probability of raw data independent of $\theta$?

If theta was different then we would have seen different combination of data, say $\theta$ is 1, (Biased towards heads) we would observe Y=1,1,1,1 for 4 tosses.

$\endgroup$

1 Answer 1

1
$\begingroup$

Intuitively, if you know that there are $y$ heads in $n$ tosses, no arrangement of 1s and 0s are superior to another, so each of the arrangements has the same probability, e.g. $$P(X^3=(1,1,0))=P(X^3=(1,0,1)=P(X^3=(0,1,1)))$$

which makes $$P(X^n|Y=y,\theta)=\frac{1}{n\choose y}$$ since there are $n \choose y$ equally likely situations. We can also find the same result using Bayes Rule, but intuition is simpler I suppose.

$\endgroup$
2
  • $\begingroup$ I see so it is a probability of "the arrangement" in an outcome, right? Not the probability of an "outcome"? I do suppose that once we know that the outcome is say 2 heads out of 3 tosses, the precedence or sequence of the outcome is independent of $\theta$ so technically the "information" is a measure of "sequence"? $\endgroup$ Commented Apr 3, 2020 at 16:34
  • $\begingroup$ correct! it's the probability of a specific arrangement, and independent of $\theta$. $\endgroup$
    – gunes
    Commented Apr 3, 2020 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.