1
$\begingroup$

Let $X$ be a continuous random variable with support $\mathcal{X}$ and density $f(x)$. Suppose I'm interested in constructing a consistent estimator of $E(X)$ using $n$ i.i.d. observations $(X_1,..., X_i, ...,X_n)$ with $X_i\sim X$.

As a first simple thought, I would consider $$ \hat{\mu}_n=\frac{1}{n} \sum_{i=1}^n X_i $$ which is a consistent estimator of $E(X)$ as $n$ goes to infinity, under some conditions.

Suppose I want to complicate my life and take the definition of $E(X)$ which is $$ E(X)=\int_{x\in \mathcal{X}} x f(x)dx $$ At this point, I could consider a kernel estimator for $f(x)$ (let me denote it by $\hat{f}_n(x)$) which is consistent for $f(x)$ under certain assumptions. In turn, I could compute $$ \tilde{\mu}_n=\int_{x\in \mathcal{X}} x \hat{f}_n(x)dx $$ which is a consistent estimator of $E(X)$ as $n$ goes to infinity, under some conditions.

If the above is correct, it seems to me that $\hat{\mu}_n$ and $\tilde{\mu}_n$ achieve the same objective, although under different (more or less stringent) set of conditions. Is this true?

$\endgroup$

1 Answer 1

1
$\begingroup$

They do achieve the same objective, and we can make the comparison even clearer. Let $\hat F_n$ give the empirical CDF and let $\hat f_n$ be the corresponding pmf that puts a mass of $1/n$ on each observed value (which are almost surely unique here). Then the first moment of $\hat F_n$ is $$ \int x \,\text d \hat F_n = \sum_{x \in \{x_1,\dots,x_n\}} x \hat f_n(x) = \bar x_n \to_p \int x \,\text d F = \text E[X] $$ so even $\hat \mu_n$ can be viewed this way.

I think this is a consequence of how we can think of moments as being functionals of probability measures via $\nu \mapsto \int x \,\text d\nu$ so it is less surprising that we can approximate a moment of $P$ (the probability measure with CDF $F$) by using moments of estimators of $P$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.