Imagine that Alice is flipping a coin with unknown bias $\theta$ and reporting the results to Bob, who is conducting Bayesian inference. If Bob begins with a uniform prior, then his posterior distribution after observing H heads and T tails will be a Beta(H-1,T-1). So far, so good.

Now suppose that instead of reporting the results directly to Bob, Alice observes the $i$th flip and then turns the coin over with probability $p_i$. She then reports the state of the coin, along with the probability $p_i$ that the original toss was altered. From this data, Bob can derive the probability $q_i$ that the $i$th toss was originally Heads. (Briefly, $q_i$ is equal to $1-p_i$ if the reported bit is Heads, and $p_i$ otherwise.) Note that if the error probabilities are zero, then the scenario reduces to the classical setting described in the previous paragraph.

If Bob does proper Bayesian inference from a uniform prior as before, he should have a posterior density of the form:

$$P(\theta|\{q_i\})= \frac{\prod_i (q_i\theta + (1-q_i)(1-\theta))}{\int_0^1\prod_i (q_i\theta' + (1-q_i)(1-\theta'))d\theta'}.$$

Note, again, that if the error probabilities are zero, then each $q$ must be either zero or one and we recover the Beta distribution. We can think of this distribution as a generalization of the Beta distribution to the scenario where observations of a Bernoulli r.v. are made under conditions of uncertainty.

My questions are:

1) Has anyone studied this before? The Beta distribution has apparently been generalized in approximately 30 different ways, but I haven't seen anything that resembles the distribution I derived.

2) Isn't there something odd about the fact that there are no sufficient statistics for the data $\{q\}$? In the limiting case of the Beta distribution where $q_i \in \{0,1\}$ for all $i$, $N$ and $K=\sum_i q_i$ are jointly sufficient to parametrize the distribution. Once we allow $q$ to take on continuous values, the numerator expands into a sum of elementary symmetric polynomials in $\{q_i\}$, and there are no longer any sufficient statistics-- we must remember the value of each $q$.

Thanks in advance for any pointers or comments.

  • 1
    $\begingroup$ Could you explain in a few words what is a binary symmetric channel ? $\endgroup$ – Stéphane Laurent Aug 18 '15 at 18:14
  • 1
    $\begingroup$ @StéphaneLaurent Sure, I edited the question to remove the reference and clarify the scenario instead. Please let me know if it's still unclear. $\endgroup$ – Patrick O'Neill Aug 18 '15 at 18:25

answer to your second question: it's quite natural that you do not have have sufficient statistics (or more precisely, non-trivial sufficient statistics: the vector of all observations is always sufficient): that's the general regime. The exceptional case is when you have a finite dimensional sufficient statistic whose dimension doesn't grow with the number of observations. That only happens when you have a conjugate family for all your likelihoods


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.