I have data points located at $\mathbf{x}_i$ and I would like to a find quick and dirty way to calculate their probability of occurring (not the pdf) using kernel density estimation. Formally speaking, I know I must take the definite integral within the neighborhood of the points to get an actual probability. But that is not feasible, due to the large number of points and high dimensionality. Does anyone have an informal way to do this?


The problem here is that your question is contradictory. You are using a KDE with a continuous kernel, which means that you are estimating using a continuous distribution. For a continuous distribution, the probability of any outcome is zero (see e.g., here and here), so we usually measure by the probability density instead. However, you say that you want the probability of the point, not its density. You also make it clear that you want the probability of the individual point, not the probability of a neighbourhood containing that point.

Under these requirements, the estimated probability of the outcome is zero. This is not helpful, which is why we measure outcomes in a continuous distribution by their probability density instead of their probability.


I'm answering my own question, and would like to know what people think of it. In general, the pdf is given by

$f(\mathbf{x})=\frac{1}{N} \sum_i K(\mathbf{x}-\mathbf{x}_i)$

where $N$ is the number of points and $K$ is a function that integrates to 1. Also, let's say that $K$ is constant for all distances less than $r$, and zero outside it ($K$ is a ball). What if we approximate the probability of finding a point within the $r-$neighborhood of $ \mathbf{x}_i$ as:

$p(\mathbf{x}_i) = \frac{f(\mathbf{x}_i)}{K(\mathbf{0})}$

where $K(\mathbf{0})$ is the density at the center of the kernel?

Let's consider two limiting cases:

1) All points are identical: $p(\mathbf{x}_i)=1$.

2) All points are far away from each other: $p(\mathbf{x}_i)=1/N$.

This method gives the correct behavior in these two cases. Of course, the probability depends on the kernel choice. This is just a quick and dirty method, but please comment if you have any issues with the approximation, or ways to improve it without integrating.

  • 3
    $\begingroup$ Your question asks for probabilities but you aren't computing probabilities here. (In relation to your question - you should also note that density is distinct from probability) $\endgroup$ – Glen_b Nov 29 '17 at 2:00
  • $\begingroup$ I believe I am. I want to convert from a probability density $f$ to an approximate probability of finding a point at $x_i$ (or pretty close to it). I made some edits to clarify. $\endgroup$ – user3433489 Nov 29 '17 at 2:02
  • 4
    $\begingroup$ The probability of getting a point from a continuous density is 0. The probability of getting "close to it" requires a specific definition of "close to"; you'll need to explain in your question what is actually required. $\endgroup$ – Glen_b Nov 29 '17 at 3:32
  • $\begingroup$ OK I added the notion of an r-neighborhood $\endgroup$ – user3433489 Nov 29 '17 at 12:03
  • 1
    $\begingroup$ No, that's not right. It happens to be true for a simple kernel and is always false for non-simple kernels with a maximum at $0$ (which characterizes just about every kernel in use). $\endgroup$ – whuber Sep 11 '18 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.