2
$\begingroup$

I am trying to estimate the value of a parameter by equating variance from a distribution to the sample variance... i.e. using method of moments estimation. Would it better to use the variance formula with $1/(n-1)$ in the denominator or $1/n$? Why is one better than the other?

thanks

I'll give an example: say I have been told Random variables from the 'geometric distribution' have a $\sum_{1}^n(X_i - \bar{X})^2$= 10. Then in order to estimate the parameter 'p' where $\frac{1-p}{p^2}$ is the variance of the geometric distribution.

do I in this circumstance use: $\frac{10}{n-1}=\frac{1-p}{p^2}$ and then given 'n' solve quadratic for p. or should I be using $\frac{10}{n}$ instead on the LHS.

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ How do you define best? You need to specify that to get an answer. In a different context where you want to choose a method to estimate variance, if you make the criterion minimum variance unbiased you would use the estimate with n-1 in the denominator. If the criteria is minimum mean square error, dividing by n could be best. In the case of a normal distribution this would be the mle. $\endgroup$ – Michael R. Chernick Sep 29 '12 at 12:59
  • $\begingroup$ I now specified a particular context. I've been reading up on variance in wikipedia and there is mention that unbiased variance may be appropriate in the situation i described...though i'm still unsure. $\endgroup$ – plantt Sep 29 '12 at 13:34
1
$\begingroup$

Regarding the method of moments and you question you can find the answer here in Wikipedia. When given a family of distributions where the distribution is determined by the value of one or more unknown parameters you can take the non central moments and given that they are a function of the unknown parameters solve k equations in k unknowns where the k equations equate the first k non central moments to the function of the parameters that they are equal to. Use the sample estimate for the moments and then solve for the parameters. If you have k parameters you have to have a case where the first k moments exist to solve. The sample moments are for a sample X$_1$, X$_2$,...,X$_n$ given by

jth moment: ∑X$_i$$^j$ /n where i runs from 1 to n and j can be 1, 2, ...,k.

So you don't need to compute a sample variance and you don't need to consider the issue of choosing between n and n-1. The method of moments was devised by Karl Pearson prior to Fisher's development of maximum likelihood. When the mle exists (regular cases) it has efficiency advantages over the method of moments. The Wikipedia link explains the pros and cons of using method of moments. The method of moments cannot be applied if the first moment doesn't exist even when k=1. But the mle may still exist and be efficient.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Say i do not know what the raw moments are... but I only know that $\sum_{1}^n(X_i - \bar{X})^2$= 10 (I know in practice raw moments are easier to calculate than the previous expression...but assuming we do not know the raw moments). In this circumstance I must choose between 1/n or 1/(n-1). Then I think, I ought to choose 1/(n-1) since it gives sample variance. $\endgroup$ – plantt Sep 30 '12 at 10:24
  • $\begingroup$ The actual moments are generally not know. The idea from the method of moments is that you use sample estimates in the fomula that relates the non-central moments to the parameters. If you can write teh central moments as a function of the parameters you should be able to with the non-central moments as well. Note that sample moments can always be estimated even when the corresponding population moment doesn't exist (e.g. you can apply the usual expressions for the sample mean and sample variance regardless of what the population distribution is). $\endgroup$ – Michael R. Chernick Sep 30 '12 at 11:29
  • $\begingroup$ The method of moments cannot be used in that case though because there is no formula for the nonexistent moments as a function of the parameters of the population's distributional family. $\endgroup$ – Michael R. Chernick Sep 30 '12 at 11:31
  • $\begingroup$ I think what you really are asking is whether or not when the variance exists for a family of distributions is it better to use the mle for the variance which uses n in the denominator or the unbiased estimate which differs because n-1 is used in place of n in the denominator. The context of applying it in the method of moments estimates for the parameters just muddies the waters if this is the real issue. The question of unbiased estimate vs mle comes up a lot and has been discussed on this site. $\endgroup$ – Michael R. Chernick Sep 30 '12 at 11:36
  • $\begingroup$ In elementary statistics classes you are taught the virtues of unbiasedness and the use of the unbiased estimator for the variance gets emphasized. The mle is often not discussed because it may involve a higher level of mathematics then is required for the course. Certainly knowledge of calculus is imprtant and advanced probability helps in understanding the optimal properties of the mle which involve asymptotic theory. But the emphasis on unbiasedness and the use of n-1 is a mistake. Bias estimators can be accurate and asymptotically unbiased. $\endgroup$ – Michael R. Chernick Sep 30 '12 at 11:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.