4
$\begingroup$

Two years ago I answered the question: What are the odds in Scrabble of not being able to make a legal move on opening?

This question is asking for the value of the parameter in a Bernouilli distribution. A player is dealt a hand of letter tiles; then either they can form a word, with probability $1-p$, or they cannot form a word, with probability $p$. We want to find $p$.

I wrote a quick program that worked as follow:

  • Repeat 50 times:
  • Draw 1000 hands of Scrabble, and count how many of those 1000 hands cannot form a word.

I end up with a list of 50 frequencies of failure: [4/1000, 2/1000, 8/1000, 4/1000, 5/1000, ...].

An obvious estimator of $p$ is the average of those 50 values. But what is the error margin around this value?

In my answer, I simply calculated the average 5.8/1000 and standard deviation 2.7/1000 of those fifty values, and concluded:

The result is a $0.58\% \pm 0.27\%$ probability that no word can be formed.

Later, someone came up with combinatorics arguments to reduce the search space and was able to run an exhaustive search, ending with the conclusion that the exact value of $p$ was $\frac{91,595,416}{16,007,560,800}$ or 0.5722%.

As you can see, my estimation of $p$ was pretty accurate: 0.58% instead of 0.5722%. However, my standard deviation was disappointingly huge; writing $p = 0.58\% \pm 0.27\%$ looks very imprecise.

  • What better confidence interval could I have given? Is there a concentration inequality particularly well-suited for this setting?
  • Was I correct to split my 50000 hands into 50 experiments of 1000 hands? What would be a better way to use these 50000 i.i.d Bernoulli trials?
Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ It is more common to use a confidence interval which is something like $\pm 1.96$ or two standard deviations rather than one $\endgroup$
    – Henry
    Commented Sep 1, 2022 at 13:36
  • $\begingroup$ @Henry Two standard deviations would be even more pessimistic than one standard deviation, and my observation here is that one standard deviation was already very pessimistic (the exact error between my estimate and the exact value was -0.0078, yet I gave the error margin as $\pm 0.27$, which is 30 times larger than needed) $\endgroup$
    – Stef
    Commented Sep 1, 2022 at 13:41

1 Answer 1

Reset to default
6
$\begingroup$

It would make more sense to say you had $50000$ attempts and $290$ successes. You have no reason to split it in the way you did, and by doing so you lost the useful information that each experiment had $1000$ attempts

This gives the same estimate of $\hat p = 0.0058$ or $0.58\%$

There are several possible ways of calculating a binomial proportion confidence interval, but you might end up with something like $[0.0051,0.0065]$ or $0.58\% \pm 0.07\%$ which is a lot narrower than your batching suggested

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for the answer and reading material! $\endgroup$
    – Stef
    Commented Sep 5, 2022 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.