Let $x_i$ be independent Bernoulli random variables with unknown success probabilities $p_i$. I want to estimate the probabilities $p_i$ depending on a number of Bernoulli trials made for each $x_i$. With rising $i$ the number of trials made for each variable ($N_i$) gets smaller.

How do I best determine the minimal $N$ for making a meaningful statement about the change in $p_i$ with rising $i$?

As the question may be unclear, here is a minimal example of the data I am talking about:

10 trials were made for variable $x_0$
7 trials were made for variable $x_1$
3 trials were made for variable $x_2$
1 trial was made for variable $x_3$

$p_i$ is unknown and may or may not be the same for $x_0$ to $x_3$. My assumption is, that there may be some clear correlation between each $p_i$ like $p_0 > p_1 > p_2 > p_3$. The problem is that I can't consider $x_3$ as only one trial was made for this variable. But can I consider $x_2$?
In the actual experiment there are 270 variables with a maximum of 99 trials.

  • $\begingroup$ The question is unclear. You're stating that you have a large (countable?) number of random variables, and for each of these you want to estimate its $p_i$? Or is there just a single success probability $p$ that you are trying to estimate? Are the trials to be split amongst the various $x_i$? $\endgroup$ – MotiN Nov 17 '16 at 11:39
  • $\begingroup$ I updated my question. I hope my problem is clear now. $\endgroup$ – Felix Engelmann Nov 17 '16 at 13:27
  • $\begingroup$ Is $n$ the number of variables? or the total number of trials? $\endgroup$ – MotiN Nov 17 '16 at 13:36
  • $\begingroup$ The total number of trials per variable. (I renamed $n$ to $N$ which I think is more common for the number of trials?) $\endgroup$ – Felix Engelmann Nov 17 '16 at 13:49

See Hoeffding's inequality and Chernoff's bound here for different ways you can bound the $p_i$ values as dependent on $N_i$ (note that the estimate would just be the empirical average, but your confidence in how near that is to the actual value would improve). Note that as your $N_i$ values decrease, the estimate is less accurate (at the same confidence level). In order to make a statement regarding your certainty that $p_i > p_{i+1}$ would thus be dependent on both $N_i$ and $N_{i+1}$.

| cite | improve this answer | |
  • $\begingroup$ Thanks for your answer! I must honestly say that I don't understand the two equations as mathematics and statistics are not my strongest fields.. I therefore can't really accept your answer as the answer that solved my question as I just don't know if it does. The good news: I managed to solve my problem in another way that may not be the most accurate but is sufficient for the report I'm currently writing. Nevertheless, thanks for your time and help! $\endgroup$ – Felix Engelmann Nov 18 '16 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.