# Kernel bandwidth in Kernel density estimation

I am doing some Kernel density estimation, with a weighted points set (ie., each sample has a weight which is not necessary one), in N dimensions. Also, these samples are just in a metric space (ie., we can define a distance between them) but nothing else. For example, we cannot determine the mean of the sample points, nor the standard deviation, nor scale one variable compared to another. The Kernel is just affected by this distance, and the weight of each sample:

$$f(x) = \frac{1.}{\sum weights_i} * \sum\frac{weight_i}{h} * Kernel(\frac{distance(x,x_i)}{h})$$

In this context, I am trying to find a robust estimation for the kernel bandwidth $h$, possibly spatially varying, and preferably which gives an exact reconstruction on the training dataset $x_i$. If necessary, we could assume that the function is relatively smooth.

I tried using the distance to the first or second nearest neighbor but it gives quite bad results. I tried with leave-one-out optimization, but I have difficulties finding a good measure to optimize for in this context in N-d, so it finds very bad estimates, especially for the training samples themselves. I cannot use the greedy estimate based on the normal assumption since I cannot compute the standard deviation. I found references using covariance matrices to get anisotropic kernels, but again, it wouldn't hold in this space...

Someone has an idea or a reference ?

• If you can measure distance, then you can measure a mean. Is that right? I might say "I am using cosine distance for words" so a "mean word doesn't really have much meaning", but I don't see why it couldn't still be computed. You could say that you are in an ordinal space, so the mean is not continuously valued. Why is the mean undefinable? Apr 12, 2016 at 16:22

One place to start would be Silverman's nearest-neighbor estimator, but to add in the weights somehow. (I am not sure exactly what your weights are for here.) The nearest neighbor method can evidently be formulated in terms of distances. I believe your first and second nearest neighbor method are versions of the nearest-neighbor method, but without a kernel function, and with a small value of $k$.