I have some data x[i] that differ from the true values by random measurement errors cx. Hence one can write
x[i] ~ N(mu,cx)
My question is: how can I determine mu knowing only x[i] and cx?
Thanks in advance
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
I assume, you know your data is normally distributed and you only need to find an estimate for the expectation and the variance. A possible estimator is the so called Maximum Likelihood Estimator (MLE). If you know the distribution but not the parameters of your data, it finds the parameters of the distribution from which your data was most likely drawn.
For a normal distribution, the MLE for the expectation is the sample mean and the empirical variance for the variance. Please note, the empirical variance defined as $\frac{1}{n}\sum(x_i - \overline{x})$ is a biased estimator. However, R gives you the unbiased sample variance. In order to obtain the MLE estimator for the variance, you need to rescale the output of the function var() by $\frac{n-1}{n}$. See the sample code:
x <- 1:20 #sample data: not normal; only for purpose of presentation
n <- length(x)
m <- mean(x) #sample mean
sv <- var(x) #sample variance
mle_var <- sv*(n-1)/n #MLE variance
answered Jan 14, 2015 at 16:41
-
$\begingroup$ If you want to recommend MLE, you shouldn't be concerned about bias. If you're concerned about bias, you shouldn't be using MLE. All in all, intertwining these two estimators seems to muddy up the explanation more than it clarifies what is going on. $\endgroup$– whuber ♦Commented Jan 14, 2015 at 17:17
-
$\begingroup$ @whuber can you please better explain your comment? Thanks $\endgroup$ Commented Jan 15, 2015 at 13:56
-
$\begingroup$ MLE is a criterion for choosing a procedure. It does not use bias in any way. Lack of bias is another criterion for choosing procedures. The two approaches can yield different procedures. Thus, recommending MLE and then suggesting that its result be adjusted for bias uses neither one nor the other methods, vitiating almost all the characteristics of both that make them useful. That leaves you making a recommendation that appears to be ad hoc and without theoretical basis. $\endgroup$– whuber ♦Commented Jan 15, 2015 at 15:18
$\begingroup$
$\endgroup$
4
The short (and boring) answer is that you can use the sample mean $\frac{1}{n} \sum\limits_{i=1}^{n} x_{i} $ to get the "right" estimate of mu.
The longer (and much more interesting) answer is that you can use the sample mean as the best estimate of the population mean in a normal distribution with known variance because the sample mean is "sufficient" for the population mean in such a setting.
Sufficient statistics are functions of the data intended to estimate population parameters such that no other functions of the data provide additional info about those parameters. It's pretty easy to show that this is the case for the sample mean in a normal with fixed variance by using what's known as the factorization theorem.
Further, there are particular kinds of sufficient statistics (complete sufficient stats and minimal sufficient stats) that give your statistic other properties you may be interested in and, because the normal distribution is an exponential family, your particular statistic will have these nice properties.
I basically threw a bunch of jargon at you hoping some of it would be interesting and you would google/look on here for fuller explanations. Let me know if you want more, I'd be happy to help.
-
$\begingroup$ Logically, your argument would apply just as well to, say, the cube of the sample mean: that, too, is a sufficient statistic for the mean. Why, then, isn't the cube of the sample mean as good an estimator of the population mean as the sample mean itself is? Although the answer is intuitive, the point is that your answer is missing some crucial element: sufficiency is not the whole part of an explanation of how or why the sample mean really is "best." $\endgroup$– whuber ♦Commented Jan 14, 2015 at 17:20
-
$\begingroup$ @whuber As I gave it you're right, sufficiency in and of itself is not enough to talk about best. But since Lehmann-Scheffe tells us that a function of a complete sufficient statistic which is unbiased for the parameter we want to estimate is the UMVUE, I'd think that sufficiency can come pretty close. To answer your (definitely rhetorical question) for clarity, it's because the square and the cube won't be unbiased. $\endgroup$ Commented Jan 14, 2015 at 17:40
-
$\begingroup$ That is much more to the point: it starts giving some real support to your answer. In contrast to other answers that could be offered (along the lines of "use this" or "use that" favorite method), the reference to UMVU introduces the idea that there are principles for comparing estimators. Elaborating on that would channel the discussion into whether and to what extent one would prefer estimators that are unbiased, for instance, and would raise interesting questions about whether biased estimators could perform better (in terms of smaller variance or some other loss). $\endgroup$– whuber ♦Commented Jan 14, 2015 at 17:48
-
$\begingroup$ I'll post an edit later! Thanks for the direction. $\endgroup$ Commented Jan 14, 2015 at 18:12