# Laplace Rule of Succession with linear constraint

Assume we label both sides of a coin as $x=\{1,2\}$ instead of $\{\text{Heads}, \text{Tails}\}$. We flip the coin 7 times and record frequencies of $f=\{3,4\}$. According to Laplace Rule of Succession (link) the probability of flipping $x=1$ on the next flip is

$$P(X_8=1 \mid X_1,X_2,\dots,X_7)={3+1 \over 7+2} = \frac 4 9$$

This was able to numerically confirm by Monte Carlo by picking $p_1$ uniformly random, with $p_2=1-p_1$, then calculating the likelihood of obtaining $f=\{3,4\}$, and calculating the expected value of $\{p_1, p_2\}$ over 1,000,000 simulations. Numerically agreed to $\{.4444, .5555\}$

I then thought, what if the actual frequency wasn't given, but that the total after $7$ flips was given instead; what would be the probability of $P(X_8=1 \mid \text{total}_7=11)$?

There are $6$ ways to total $11$ with $x=\{1,2\}$, being

$$f=\{\{1,5\},\{3,4\},\{5,3\},\{7,2\},\{9,1\},\{11,0\}\}$$

According to my understanding of Laplace not only are ${p_1,p_2}$ uniform but also each one of the six combinations are equally likely. Redid the Monte Carlo this time instead of $\{3,4\}$ as before I randomly picked one of the six combinations as a possible frequency count before calculating the likelihood for that simulation. After $1{,}000{,}000$ simulations obtained expected value of probabilities as $\{.764, .236\}$

This is significantly different than before. The first number seems quite high.

Is that the correct way to handle Laplace Rule of Succession with unknown actual frequencies? All possible combinations that could produce those constraints are equally likely? (Generalized Example: dice and we were given three linear combinations instead of actual frequencies after rolling it N times)

(The expected value of the frequencies, instead of the probabilities, was $f=\{8.5147, 1.2427\}$ )

Interestingly, although not correct according to Laplace, there are two equations and two unknown $p_1+p_2=1$ and $7(1\cdot p_1 + 2\cdot p_2) =11$ which produces $\{0.4286, 0.5714\}$ which is a lot closer to the original values. This was tangential as Laplace integrated over all possible cases.

EDIT:

Just realized this was a bad example; as if total was 11 out of 7 flips it has to be f={3,4} for x={1,2}. However, the generalized question is the same: How would you solve for Rule of Succession when linear combinations of the frequencies are given and not the actual frequencies themselves?

An example could be rolling a regular die and knowing only three linear combinations out of 100 rolls. The generalized rule of succession (link) is valid when the actual frequency count is known.

One method to solve for the probabilities with linear constraints is Principle of Maximum Entropy (link), but that is different technique/reasoning than Laplace of integrating over all possibilities. Laplace is wondering about the next flip/roll/event taking into account all possibilities and likelihoods, whereras Maximum Entropy prescribes a specific probability from an optimization problem. Or are they equivalent in some sense?

• If you don't encode "success" as 2 and "failure" as 1, but use the standard 0/1 instead, then the "total count" would be the same as the frequency of successes. I think this count variable is too much dependent on the notation you use for success and failure. Commented Feb 17, 2017 at 23:11
• Where did "$7+2$" come from in the denominator? Perhaps you intended $7+3$?
– whuber
Commented Feb 17, 2017 at 23:49
• Correct; the reason for 1/2 instead of 0/1 was so that total count wouldn't equaI frequency of success. It adds more uncertainty as a linear combination of {1,2} is known but not any individual frequencies. This would not be the case for {0,1} Commented Feb 18, 2017 at 1:01
• @whuber The 2 in the denominator came from Laplace's formula directly in the wikipedia page. There are m=2 categories (succes,failure) so the denominator is n+m. I also just added an edit to clarify question being asked. Thanks. Commented Feb 18, 2017 at 1:03

I think Laplace's rule of succession does not apply directly when the probability $p_1$ is unknown. In that case one starts with a prior, for instance $p\sim\text{U}(0,1)$ which may be called Laplace's prior since he used such a prior for male/female births. Whatever the observables are, one then compute the posterior on $p_1$.
$$f=\{\{1,5\},\{3,4\},\{5,3\},\{7,2\},\{9,1\},\{11,0\}\}$$
According to my understanding of Laplace not only are $(p_1,p_2)$ uniform but also each one of the six combinations are equally likely.