Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 13 down vote favorite
4

In a setting where one observes $X_1,\ldots,X_n$ distributed from a distribution with density $f$, I wonder if there is an unbiased estimator (based on the $X_i$'s) of the Hellinger distance to another distribution with density $f_0$, namely $$ \mathfrak{H}(f,f_0) = \left\{ 1 - \int_\mathcal{X} \sqrt{f(x)f_0(x)} \text{d}x \right\}^{1/2}\,. $$

share|improve this question
4  
So f0 is known and fixed. But is f known or from a parametric family or are doing this in a nonparametric framework with all you know about f coming from your sample? I think it makes a difference when attempting an answer. – Michael Chernick Jun 1 '12 at 10:16
3  
@MichaelChernick: assume all you know about $f$ is the sample $X_1,\ldots,X_n$. – Xi'an Jun 1 '12 at 10:57
2  
I do not think it has been calculated (if there exists). If there exists, then AIC has a lost brother. – user10525 Jun 1 '12 at 12:49
1  
I think I am in agreement with Procrastinator. I haven't got any idea about how this could be done nonparametrically. You could use a kernel density estimate of f. But the density surely could not be unbiased for all x. So how could you possibly choose a kernel that would make that function an unbiased estimator of the distance. This is even worse of a problem if f and f0 have unbounded range because your data gives you no information on the very extreme tails which could still play a role in the calculation of the integral. I can't prove that it is impossible but I think it is! – Michael Chernick Jun 1 '12 at 14:48
3  
An attack on this problem looks feasible if you assume $f$ and $f_0$ are discrete. This leads to an obvious estimator (compute the Hellinger distance between the EDF and $f_0$). Bootstrapping (theoretically, not via simulation!) will give us a handle on the possible bias as well as a way to reduce (or even eliminate) the bias. I hold out some hope to succeed with the squared distance rather than the distance itself, because it is mathematically more tractable. The assumption of a discrete $f$ is no problem in applications; the space of discrete $f$ is a dense subset anyway. – whuber♦ Jun 1 '12 at 16:00
show 7 more comments

1 Answer

up vote 7 down vote accepted

I don't know how to construct (if it exists) an unbiased estimator of the Hellinger distance. It seems possible to construct a consistent estimator. We have some fixed known density $f_0$, and a random sample $X_1,\dots,X_n$ from a density $f>0$. We want to estimate $$ H(f,f_0) = \sqrt{1 - \int_\mathscr{X} \sqrt{f(x)f_0(x)}\,dx} = \sqrt{1 - \int_\mathscr{X} \sqrt{\frac{f_0(x)}{f(x)}}\;\;f(x)\,dx} $$ $$ = \sqrt{1 - \mathbb{E}\left[\sqrt{\frac{f_0(X)}{f(X)}}\;\;\right] }\, , $$ where $X\sim f$. By the SLLN, we know that $$ \sqrt{1 - \frac{1}{n} \sum_{i=1}^n \sqrt{\frac{f_0(X_i)}{f(X_i)}}} \quad \rightarrow H(f,f_0) \, , $$ almost surely, as $n\to\infty$. Hence, a resonable way to estimate $H(f,f_0)$ would be to take some density estimator $\hat{f_n}$ (such as a traditional kernel density estimator) of $f$, and compute $$ \hat{H}=\sqrt{1 - \frac{1}{n} \sum_{i=1}^n \sqrt{\frac{f_0(X_i)}{\hat{f_n}(X_i)}}} \, . $$

share|improve this answer
3  
@Zen: Good point! I consider this answer as the answer because it made me realise $H$ sounds very much like a standard deviation, for which there exists no unbiased estimator. As for the variance of $\hat H^2_n$, no worries: $\mathbb{E}[(\sqrt{f_0(X)/f(X)})^2]=1$ implies that this estimator has a finite variance. – Xi'an Oct 19 '12 at 5:42
1  
Thanks for the clarification about the variance of the estimator, Xi'an! – Zen Oct 19 '12 at 13:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.