1
$\begingroup$

Suppose we can always generate a sample $\{X_1, \dots, X_n\}$ of size $n$ for a discrete distribution, such that its empirical pmf is the same as the true pmf. One of the reason that we can have such assurance is when $X_i$ is a transform of a uniform discrete random variable $Y_i$, and we can generate a sample of $Y_i$ by deterministically enumerating the members of its range.

for example if the common distribution is a Bernoulli distribution with parameter $p$. If we can always ensure that $\sum_i X_i /n = p$, shall the deviation be estimated as ${\frac{1}{n} \sum_i (X_i - \bar{X}}^2)^{-1/2}$ or ${\frac{1}{n-1} \sum_i (X_i - \bar{X}}^2)^{-1/2}$. Normally we would prefer the latter because it is unbiased while the former isn't. But here we know exactly that $\sum_i X_i /n = p$. Will the former be better than the latter?

Thanks and regards!

$\endgroup$
  • 1
    $\begingroup$ It seems a stretch to call a "deterministic enumeration" a sample. $\endgroup$ – whuber Mar 28 '13 at 17:00
2
$\begingroup$

Unbiasedness is a property of a estimator applied to a random sample, but you are supposing that the sample is not random. Specifically, you assume that the sample distribution is always equal to the true distribution. For example, pmf = multinomial(1/2, 1/4, 1/4) and a sample of N=8 always comes out {a,a,a,a,b,b,c,c}. That constraint means you aren't taking a random sample; your outcome is always the same - {a,a,a,a,b,b,c,c}.

So it does not make sense to use an estimator that has good properties for random samples under your assumption. Your deterministic sample is the pmf, so you'd calculate your moments directly on the sample using the formulas for the pmf. In your variance for a Bernoulli example, you would divide by n, not n-1.

$\endgroup$
1
$\begingroup$

The reason you divide by $n-1$ is the loss of a degree of freedom when you have to estimate the mean of the distribution. If you know the mean with certainty, then I would think that you could divide by $n$ and have an unbiased estimator.

Not that this is an authoritative reference, but I'm sure it's correct: http://www.mathsisfun.com/data/standard-deviation-formulas.html

$\endgroup$
  • $\begingroup$ Thanks! Not sure if you have noticed, I added a fact to the end of the first paragraph, for the case when we can get the exact empirical pmf same as the true one. $\endgroup$ – Tim Mar 28 '13 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.