3
$\begingroup$

Sorry, I'm a huge statistics noob. I'm looking to find the probability that a certain event occurs in a binomial experiment. I take a few thousand samples per test, and then calculate the probabilities. The problem is that the experimental probabilities keep varying by a small amount. So I've started plotting the experimental probabilities as a function of sample size. When I look at the plots for these "cumulative averages" the trend I observe is that the experimental probabilities vary wildly at first and then taper off and oscillate around the true probability. However, I can't get enough samples per test to observe a straight line-like convergence.

My question: can I average out these cumulative averages I've plotted for a more precise estimate of my true event probability? And if I can, would it be better to use a weighted average - like give more weight to probabilities using a higher number of samples and lower weight to probabilities using a lower number of samples? Or maybe even take the average of only the experimental probabilities that begin to converge very finely. Thanks

Improve this question
$\endgroup$
5
$\begingroup$

Major edit: I misunderstood the OP's question. The question is about averaging the average of the first $n_1$ trials, and the first $n_2$ trials ($n_2>n_1$), and the first $n_3$ trials ($n_3>n_2$), and so on, where all the data is in one experiment. That is, where the first observations appear in the first average and the second average and the third average ... but the last observations only appear in the last of them.

Short answer: don't do that!

Longer answer: Just look at the simple average of all the data - presumably your final average. It has all the information in it already.

Averaging it with any earlier average actually

(i) dilutes the information it contains - makes it less informative, and
(ii) makes it hard to compute the standard error of that proportion, since you lose independence.

There's no good reason to do this, and lots of good reasons to avoid it.

Original answer about independent experiments --

If you're averaging separate experiments:

If the binomial parameter (the probability per trial) is constant over all the things being averaged, then yes, you can average the averages, and yes, it should be a weighted average.

That's actually the same as taking the whole of the data as one big binomial and taking a single average over the lot.

Edit:

Let's say you observe two sets of events (call each set 'an experiment'). In the first experiment, you get the event of interest 35 times in 1130 trials and in the second experiment you get the even 25 times in 1015 trials. This is the same as observing (1130 + 1015) trials and getting the event of interest (25 + 35) times.

When you estimate the proportion as $\frac{35+25}{1130+1015}$, it's the same as working out the weighted average: $w_1\frac{35}{1130} + w_2\frac{25}{1015}$ where $w_1 = \frac{n_1}{n_1+n_2} = \frac{1130}{1130+1015}$ and $w_2 = \frac{n_2}{n_1+n_2} = \frac{1015}{1130+1015}$. So the appropriate weighted average of the two experiments is the same as treating it as one, larger experiment.

It works similarly for a bunch of experiments.

Improve this answer
$\endgroup$
  • $\begingroup$ Thanks a lot for your answer. But what do you mean by taking the whole data as one big binomial and taking a single average over the lot? $\endgroup$ – greg L Jul 4 '13 at 6:49
  • $\begingroup$ See my edit above. $\endgroup$ – Glen_b Jul 4 '13 at 8:58
  • $\begingroup$ Ohh wait I think you misunderstood me. Or my question was not properly written. I meant to ask if you could take the average of the cumulative averages within a single experiment. Can you take the weighted average of the estimated proportions using n=5,10,15,etc? $\endgroup$ – greg L Jul 4 '13 at 18:27
  • $\begingroup$ Oh, okay, I see now. No, definitely don't do that. Hang on, some edits are in order! $\endgroup$ – Glen_b Jul 5 '13 at 0:04
  • $\begingroup$ shady milkman -- now edited; please review. $\endgroup$ – Glen_b Jul 5 '13 at 0:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.