2
$\begingroup$

Suppose we have N independently drawn samples from an unknown random variable $X$. What is the best way to estimate the expectation of $X$?

For simplicity, we can assume that $X$ only returns values between 0 and 1.

The standard way to estimate the mean is just to just take the average of all the samples. But I got to thinking that this isn't optimal. For example, maybe it's better to take the median-of-averages? I.e. to split the samples into some number of equal-sized buckets, compute the average of each bucket, and then take the median of the averages?

Subquestion: what is the best way to estimate the mean assuming $X$ is a normal random variable with standard deviation 1? Even for this goal I'm not sure which is better: median, average, or something else.

Update: After thinking about @Nick's comment, I think there might be no "best" estimator for the mean. In particular, even for normal distributions, I think that taking the sample mean has advantages and disadvantages over taking the median-of-averages or just the median. Overall, I think median-based methods will give a more spread-out answer, but you'll have better certainty to be within some confidence interval than using mean, but I need to do the math to be sure of this. Maybe there are some good references on the topic?

|cite|improve this question|||||
$\endgroup$
  • 1
    $\begingroup$ Best in what sense? Note that your main question and subquestion are completely separate, as a distribution bounded by 0 and 1 cannot be normal with standard deviation 1 (regardless of what the mean of the latter is). There is much literature on how sample means can be a poor method for heavy-tailed distributions, but for arbitrary distributions on [0,1] not much can be said. For normal distributions there is no reason not to use the sample mean. $\endgroup$ – Nick Cox May 27 '13 at 11:54
  • $\begingroup$ @Nick Thanks for the answer! Do you have any good references on estimating the mean of various types of distributions? I don't know the stats literature, and failed to find a decent reference. Also, The main question and the subquestion are indeed separate, but the subquestion should be significantly easier. Are you sure that for a normal distribution using the sample mean is "best"? Lastly: Yes, I'm not sure what I mean by "best". Presumably, there would be one method that is clearly best. Right now I suspect there is actually no such method, even for normal distributions. $\endgroup$ – eldodo42 May 27 '13 at 14:21
  • 1
    $\begingroup$ What to read depends on your previous knowledge and mathematics level, not clear here, but just about any intermediate or higher discussion of statistical theory talks about properties of estimators (biased or unbiased, efficient, etc., etc.). As you don't expand on your intuitions of "best" it is difficult to comment on them, but if you have a normal, the sample mean does score very well as an estimator of the population mean. Literature on robust statistics covers what goes wrong when distributions are awkward, especially heavy-tailed. $\endgroup$ – Nick Cox May 27 '13 at 14:37
  • 1
    $\begingroup$ Yes, as said or implied, keywords include estimation, estimators, bias, efficiency, robust statistics. $\endgroup$ – Nick Cox May 27 '13 at 16:22
  • 1
    $\begingroup$ The usual way this is approached is in terms of the variance of estimates yielded by an estimator. $\endgroup$ – Nick Cox May 27 '13 at 17:29
1
$\begingroup$

Maybe you can also play with bootstrapping? Somehow like this: 1) draw n bootstrap-samples with replacement from your original sample of the same size as your original sample 2)calculate the mean for every bootstrap-sample and look at the distribution of these means. You can compare the mean of the generated distribution with the mean of the original sample to see if the original mean is biased.

|cite|improve this answer|||||
$\endgroup$
  • 2
    $\begingroup$ Welcome to the site, @user2317915. It's not clear that this is an answer to the OP's question. Can you expand it into an answer (eg, by explaining how bootstrapping might be a viable solution)? $\endgroup$ – gung - Reinstate Monica♦ May 26 '13 at 17:14
  • $\begingroup$ yeah, I don't quite understand the suggestion. $\endgroup$ – eldodo42 May 26 '13 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.