I have an unknown process that produces binary results. I am trying to determine if this process is a Bernoulli trial.
In the theory of probability and statistics, a Bernoulli trial (or binomial trial) is a random experiment with exactly two possible outcomes, "success" and "failure", in which the probability of success is the same every time the experiment is conducted.
I do not know the actual success rate, if the success rate of each trial is equal, or if the trials are independent from each other.
However, I can collect data from the process in sequential trials with its corresponding successful trials.
Initial set of trials yielded 5 successes out of 999 trials.
- Success 1 was on trial 196
- Success 2 was on trial 388
- success 3 was on trial 593
- Success 4 was on trial 792
- Success 5 was on trial 999
What is the ideal way to demonstrate (perhaps with test statistics) if this process is or is not a Bernoulli trial. i.e.
- I already know it outputs binary results.
- does each trial have an equal or varying probability of success?
- are the trials independent?
A computer game uses an unspecified item drop system. Most item drop systems in computer games are Bernoulli trials with equal probability of success (i.e. an equal drop chance). However, I suspect that this particular system uses a count to determine a successful drop (e.g. success after ~200 trials). I want to design an experiment that will prove that the drop system uses an equal drop chance (Bernoulli trial) or a different system (e.g. a count based system).
I had only very basic statistics lesson in college many years ago. I am very fuzzy with test statistics. I have fair understanding of Bernoulli trials and binomial distributions.
Since I am trying to determine if the process is a Bernoulli trial with an equal chance of success p, if I can demonstrate that separate cumulative trials have different p, I can prove that the drop system does not use Bernoulli trial with an equal success chance.
To do this I collected 3 sets of data. Set A was a sequential 999 trials that resulted in 5 successes as stated above. Set B was 5 x 190 trials with a gap in between them. Set C was the gap between the 5 sets of 190 trials in B. i.e.
999 trials (A) - 190 (B) - 15 (C) - 190 (B) - 11 (C) - 190 (B) - 9 (C) - 190 (B) - 1 (C) - 190 (B) - 3 (C) - 190 (B)
This sampling yielded:
- A: 5 successes in 999 trials
- B: 0 successes in 950 trials
- C: 5 successes in 39 trials
The reason it is sampled this way is because I suspected that the drop counter is between 190 and 210, and that the drop counter resets after each successful trial. Every time I obtain a successful trial in Set C, I switch back to collecting Set B.
Using Clopper-Pearson interval (a.k.a. 'exact' method), α = 0.05, I calculated the interval of which Success chance p lies in, for each set of data.
- Set A: Success rate PA: 0.00163 - 0.01164, 95% confidence
- Set B: Success rate PB: 0.00003 - 0.00388, 95% confidence
- Set C: Success rate PC: 0.04297 - 0.2743, 95% confidence
PC ≠ PA
PC ≠ PB
I conclude that the drop system observed is not a binomial experiment with equal success chance.
Is this an acceptable solution? If not, what is the correct method to approach this problem?
I've read through several posts related to this topic, however they are difficult for me to understand and I cannot be sure how to use those solutions on this particular question.