I am thinking of using this code in a Monte Carlo routine to generate Kernel Density Estimates for subsequent use in a Naive Bayes Classifier (see this earlier post).

The author of the code states on the above linked page that it will "recognise that the data you have provided is perfectly discrete and since discrete data does not need smoothing, the selected bandwidth should be zero" in the case of ties. However, I do not want this but can envisage that, given the parameters I intend to use, multiple repetitions in a large Monte Carlo routine will very likely produce ties in the generated data. To prevent this I am thinking of limiting the number of iterations in the MC routine and employing checks to ensure non-discrete generated data for use in the Kernel Density Estimate code, and then repeating. Perhaps this is more clearly explained thus: instead of a 100,000 iteration MC, which is highly likely to produce unwanted ties, and then one KDE, do a 100 iteration MC and one KDE and repeat this 100 and 1 routine 1000 times. This will result in 1000 separate but similar continuous KDEs which can then be averaged to produce a single "unified" continuous KDE.

Is this a valid approach? Are averaged KDEs a statistically sound methodology? The discussion section here would seem to imply that even if there are deficiencies in this approach, they wouldn't necessarily be too troublesome.

  • 1
    $\begingroup$ Here is the link to implementation in R (www-etud.iro.umontreal.ca/~botev/kde.R). I understand your question now. Mine is why you want to do that ? I would perform a MC as described to get an estimate of the error of the kde, but not to calculate a kde for NB, which uses maximum likelihood estimates of all variables per definition ! I think that performing the algorithm as described by you and calculating the kde directly from the training data (without any subsampling) will not affect the result of NB. $\endgroup$
    – steffen
    Commented Nov 19, 2010 at 10:43
  • $\begingroup$ My guess would be that since KDR is basically linear then it probably makes very little difference, but I'm not expert in this area. I don't know what effect the discreteness has on the problem. Can anyone improve on @steffen's answer/comment? $\endgroup$
    – Corvus
    Commented Feb 18, 2013 at 21:57


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.