I am trying to conduct a simulation study on a variation of the kernel density estimator. In that experiment, there is a parameter in one of my formulas that involves the value of the unknown pdf at a given point x. Therefore I need to find a consistent estimator to estimate the value of f(x), but I have no clues on where I should get started with.

Can anybody suggest an estimator of f(x) that is consistent or which family of estimators I should be looking into, particularly, for a normal distribution?

Thank you!!

  • $\begingroup$ @whuber, what do you mean by "Theoretically, a PDF has no value at any given point"? Do you mean practical value or numerical value? $\endgroup$ – Macro Jun 5 '12 at 17:10
  • $\begingroup$ @Macro, a pdf is not a function; it's actually a member of an equivalence class of functions: you can change its values arbitrarily on any set of Lebesgue measure zero. This is why Michael Chernick is careful to focus on "absolutely continuous" distributions in his reply. If that sounds like theoretical nit-picking, it is (to some extent); but the same theoretical considerations suggest that perhaps there's more underlying this question than one might guess and we should be cautious about taking it at face value. $\endgroup$ – whuber Jun 5 '12 at 17:17
  • $\begingroup$ @whuber Thanks for the advice, I modified the question a little bit according to your suggestion. It's not that deep as you thought actually :) $\endgroup$ – Vokram Jun 5 '12 at 17:23
  • $\begingroup$ Vokram, I was referring to the density: see my reply to Macro above. Re your edit: although it helps, I find it confusing that although you are studying a KDE (which estimates an entire PDF), yet you are asking for an estimator of $f(x)$. How exactly does this work? Does your KDE require some kind of initial estimate $\hat{f}(x)$ and then construct an estimator of all of $f$ from that? $\endgroup$ – whuber Jun 5 '12 at 17:24
  • 1
    $\begingroup$ @whuber yeah, kind of. It's a improved version of KDE (which is biased itself anyways), in which I need to employ a consistent estimator to get a rough estimation of $f(x)$ before the new method can be employed. I see what you mean by the pdf thing now, reminds me of the pain to get from $\mathcal{L}^{1}$ to $L^{1}$...:) $\endgroup$ – Vokram Jun 5 '12 at 17:35

Assume an absolutely continuous distribution $F$ with density $f$. Then since $f$ is the derivative of $F$ $\lim_{ e \rightarrow 0} \dfrac{F(x+e)- F(x-e)}{2e} = f(x)$. So to estimate $f$ consistently at $x$ for any fixed sample size $N$ let $p(x) = \dfrac{s}{N}$ where $s$ is the number of observations in the interval $[x-e, x+e]$. Let $e \rightarrow 0$ at a rate of $1/N$ as $N \rightarrow\infty$ Then we have that $\dfrac{p(x)}{2e} \rightarrow f(x)$ as $N \rightarrow\infty$ . So let $h(x) = \dfrac{p(x)}{2e}$. Then $h(x)$ is a consistent estimator for $f$ at the point $x$. This will work for all absolutely continuous distributions including the normal.

| cite | improve this answer | |
  • $\begingroup$ You need to be a little careful about how $e\to 0$ as $N\to \infty$. Without explicit control over the relative rates of convergence, I believe you could arrive at any value between $0$ and the correct one. E.g., letting $e=N^{-2}$ gives $p(x)\to 0$ a.s. $\endgroup$ – whuber Jun 5 '12 at 17:12
  • $\begingroup$ For e fixed and N → ∞ p(x) → [F(x+e)-F(x-e)]. So if I let e→0 at a rate slower than 1/N then p(x)/(2e) will approach f(x). $\endgroup$ – Michael R. Chernick Jun 5 '12 at 17:38
  • $\begingroup$ @MichaelChernick Thanks Michael, but can you tell me a little more about why it will work if e converges slower than 1/N? Intuitively I think that makes sense, but I would appreciate a little reminder from the theoretical side. Sorry I am quite rustic with statistics... $\endgroup$ – Vokram Jun 5 '12 at 17:57
  • $\begingroup$ f(x) =F'(x) so f(x)=lim e→0 [F(x+e)-F(x-e)]/(2e) As long as p(x) is close to the numerator p(x)/(2e) will approach f(x). But if e→0 at a rate of 1/N or faster p(x)/(2e) can go to 0 as Bill Huber pointed out. As N → ∞ p(x)→0 because the proportion in the small interval will go to 0 since F is absolutely continuous. Now p(x) is going to 0 at a rate of 1/N so if e approaches 0 at a rate faster than 1/N then p(x)/2e → ∞. If it goes at a rate slower than 1/N then it will actually go to 0. So it will converge to f(x) only if it goes at the rate 1/N. $\endgroup$ – Michael R. Chernick Jun 5 '12 at 18:13
  • $\begingroup$ @Michael, I believe that when $e\to 0$ slowly and $x$ is in the interior of the essential support of $f$, then $p(x)/(2e)$ will converge to $f(x)$, not $\infty$. This is a two-epsilon argument: when $N$ is sufficiently large, $p(x)$ will be close to $\int_{x-e}^{x+e}f(x)dx = (F(x+e)-F(x-e))/(2e)$, which converges to $f(x)$. $\endgroup$ – whuber Jun 5 '12 at 19:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.