# kernel density estimation of the log-normal distribution

I stumbled upon the following issue I cannot make sense of:

When using default choices, the KDE for a log-normal sample (green) does not look like a density that integrates to 1, compare the true density (violet):

I created this using

set.seed(1)
n <- 1e6
xax <- seq(-3,20,by=.1)

x <- rlnorm(n)
plot(density(x),lwd=3,col="seagreen",xlim=c(-3,20))
lines(xax,dlnorm(xax),lwd=3,col="palevioletred1")


This does not look obviously wrong, because it seems to produce decent results when applied to a $\chi^2$-distribution, and excellent results for a normal population.

x <- rchisq(n,2)
plot(density(x),lwd=3,col="seagreen",xlim=c(-3,20))
lines(xax,dchisq(xax,2),lwd=3,col="palevioletred1")


x <- rnorm(n,10)
plot(density(x),lwd=3,col="seagreen",xlim=c(-3,20))
lines(xax,dnorm(xax,10),lwd=3,col="palevioletred1")

• I am far from fluent in R, so as always correct me if I am wrong: The density estimation code hasn't been told and certainly does not know independently that negative values are impossible for a lognormal. In the graph, there is a spurious diffusion of mass into forbidden territory. Commented Dec 6, 2021 at 15:50
• I agree, but nor does density "know" this for the second example ($\chi^2$), where the issue is less prevalent. I believe @glen_b's answer (at least for me) spots the issue. Commented Dec 6, 2021 at 16:53
• Indeed; how much this bites will depend on the distribution. I am reminded of a recent thread about the lognormal and the gamma. Commented Dec 6, 2021 at 16:55

## 2 Answers

One thing that concerns me about that is your bandwidth is 1/3 of the distance between the points your density estimate is evaluated at.

 diff(density(x)$x[1:10]) [1] 0.2055696 0.2055696 0.2055696 0.2055696 0.2055696 0.2055696 0.2055696 [8] 0.2055696 0.2055696  vs > density(x)$bw
[1] 0.06165546


This can potentially lead to odd results.

Indeed, that seems as if it may be most of the problem.

Try density(x,n=2^14) in your code. (Actually, it looks like $2^{12}$ would do, and even $2^{10}$ is a substantial improvement.)

You can see the pink here almost entirely obscures the green.

This issue of a small bandwidth relative to the inter-evaluation-point gap* is caused by the very large sample size; because bandwidth is proportional to $n^{-\frac15}$, with large enough $n$, eventually this will happen even with Gaussian data.

*[which is the (extended) range divided by default number of evaluation points (512)]

• Thanks! Would it not be a nice kind of warning if some numerical intergration in density revealed that the integral is not close to 1? Commented Aug 26, 2015 at 15:51
• Yes, it should probably check that, but an approximate integration is easy to do with the output of density in any case (that was the first thing I checked with your example). Even aside from that, I think it should warn you if the bandwidth is smaller than the evaluation-grid distance (or if it was being clever, adjust the grid to be sufficiently fine for you). Commented Aug 26, 2015 at 22:11

indeed this is a characteristic of log-distributed distributions. Other than on linear distributed (normal, poisson, etc), when representing the DPF numerically the "columns" width is not linear. This is why the green curve looks like having an area much larger than 1. Doing many points (as in the example) reduces this error by the limit theorem.