12
$\begingroup$

I have developed a simple Kernel Density Estimator in Java, based on a few dozen points (maybe up to one hundred or so) and a Gaussian kernel function. The implementation gives me the PDF and CDF of my probability distribution at any point.

I would now like to implement a simple sampling method for this KDE. An obvious choice would of course be to draw from the very set of points making up the KDE, but I would like to be able to retrieve points that are slightly different from the ones in the KDE.

I haven't found so far a sampling technique that I could easily implement to solve this problem (without depending on external libraries for numerical integration or complex computations). Any advices? I don't have specially strong requirements when it comes to precision or efficiency, my main concern is to have a sampling function that works and can be easily implemented. Thanks!

Improve this question
$\endgroup$
3
  • 4
    $\begingroup$ This is detailed in page 5 of this document. $\endgroup$
    – user10525
    Commented Nov 15, 2012 at 18:21
  • $\begingroup$ thanks, that was useful! And simpler than I thought ;-) $\endgroup$
    – Pierre Lison
    Commented Nov 15, 2012 at 19:49
  • $\begingroup$ @user10525 the code provided is incorrect, it should be: rnorm(n, sample(dx$x, n, prob = dx$y, replace = TRUE), dx$bw) where dx is output from density function. Argument prob has to be provided because otherwise you sample uniformly. $\endgroup$
    – Tim
    Commented Dec 22, 2015 at 20:29

1 Answer 1

Reset to default
23
$\begingroup$

As mentioned by Procrastinator, there's a simple way to sample from a Kernel density estimator:

  1. Draw one point $x_i$ from the set of points $x_1$,...$x_n$ included in the KDE
  2. Once you have the point $x_i$, draw a value from the kernel associated with the point. In this case, draw from the Gaussian $\mathcal{N}(x_i,h)$ centered at $x_i$ and of variance $h$ (the bandwidth)
Improve this answer
$\endgroup$
5
  • $\begingroup$ (+1) For sharing your solution. $\endgroup$
    – user10525
    Commented Nov 19, 2012 at 10:15
  • $\begingroup$ Is $x_i$ one of the original points? If so, looks like we don't really need to construct the actual KDE at all. Just sampling from one of the original points, and $N (x_i,h)$ should suffice? $\endgroup$
    – Ram
    Commented Apr 8, 2013 at 23:19
  • $\begingroup$ Yes indeed, if you are only using the KDE distribution for sampling, you do not need to explicitly construct the PDF: the only information necessary for the sampling operation is the set of points and the bandwidth. $\endgroup$
    – Pierre Lison
    Commented Apr 9, 2013 at 6:28
  • $\begingroup$ just to add to Pierre Lison: In step 2.: For sampling from a Gaussian kernel, the bandwidth h should be taken as the standard deviation of the Gaussian distribution around the point x_i, not the variance. $\endgroup$
    – user98904
    Commented Dec 22, 2015 at 18:52
  • $\begingroup$ Wouldn't you want to sample using standard deviation 1/h or something? As written, the less likely x_i is, the more likely you are to sample another unlikely point nearby because the standard deviation of N is low. $\endgroup$
    – chris
    Commented Jul 3, 2019 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.