Comparing Kernel Density Estimate with a Normal Distribution

I am trying to compare a Kernel Density estimation with a normal distribution.

I use the following code to estimate the Kernel Density using a Epanechnikov kernel:

from sklearn.neighbors.kde import KernelDensity

X_grid = np.linspace(-5, 5, num=1000)

def silverman_bw(ts):
return 1.3643*1.7188*len(ts)**(-0.2)*min(np.std(ts), np.subtract(*np.percentile(ts, [75, 25])))

kde = KernelDensity(kernel='epanechnikov', bandwidth=silverman_bw(ts5m.logreturns)).fit(ts5m.logreturns.reshape(-1,1))
pdf = np.exp(kde5m.score_samples(X_grid.reshape(-1,1)))

and the following code to generate the Normal Distribution I want to benchmark it up against

from scipy.stats.distributions import norm

normpdf = norm.pdf(X_grid)

The problem is obvious when I plot these two in the same plot; it is impossible to compare the two distributions due to different y-scales. The red line is the Kernel Density estimate and the blue-ish the Normal Distribution. What do I do in order to be able to compare these two? I am probably missing some scaling/standardization-something, but I just can't remember the precise theory, and I am unable to find answers in my academic litterature.

• You could try plotting their ratio.
– user1566
Oct 28 '16 at 20:28

The plot wouldn't look very different if you compared standard normal distribution with normal distribution parametrized by standard deviation of $\approx 0.15$... Comparing two distributions that have standard deviations differing by a magnitude of over sixty times does not make much sense. Even if they did not differ in height of density (what is impossible), then either the distribution with greater standard deviation would be outside of your plotting area, or the one with smaller standard deviation would be just a thin line in the middle of the plot...

Normal distribution is a member of location-scale family, if $f(x)$ is a standard distribution with location $\mu=0$ and scale $\sigma=1$, then if you change it's location to some other $\mu'$ and scale to $\sigma'$, it becomes $f((x-\mu)/\sigma)/\sigma$. This means that if standard normal distribution at it's highest point $x=0$ has height of $0.3989423$, then when you change it's scale to $\sigma'$, it will get $\tfrac{1}{\sigma'}$ times higher.

• Thanks for your answer Tim. You are certainly right. I was actually under the impression that the norm.pdf(x)-function scaled the pdf using x. I've tried scaling up the normpdf manually instead by multiplying with the std.dev of logreturns and adding the mean. It does not make the distributions comparable, unfortunately. Oct 29 '16 at 7:10
• @mfvas norm.pdf lets you to provide location and scale as function parameters, moreover if $f(x)$ is a standard distribution, then when it is parameterized by scale you have to take $f(x/s)/s$. The same says it's documentation: "Specifically, norm.pdf(x, loc, scale) is identically equivalent to norm.pdf(y) / scale with y = (x - loc) / scale."
– Tim
Oct 29 '16 at 7:29
• Thanks Tim. It looks like i misunderstood how the function worked. Using the built-in normal distribution generator in matplotlib I now have a comparable plot. Oct 29 '16 at 8:23
• @mfvas I'm glad it helped. I added some comment to make it more clear why did you observed what you did.
– Tim
Oct 29 '16 at 11:13

I found a solution. It turns out, with the guidance from Tims answer, that the norm.pdf() function does not work like functions I normally work with in different programming languages and thus I needed to scale my variables differently.

Using the function from matplotlib.mlb instead, which works like I expected it to, I now have comparable graphs:

import matplotlib.mlab as mlab

def silverman_bw(ts):
return 1.3643*1.7188*len(ts)**(-0.2)*min(np.std(ts), np.subtract(*np.percentile(ts, [75, 25])))

kde = KernelDensity(kernel='epanechnikov', bandwidth=silverman_bw(ts5m.logreturns)).fit(ts5m.logreturns.reshape(-1,1))
pdf = np.exp(kde5m.score_samples(X_grid.reshape(-1,1)))

f, ax = plt.subplots()
ax.set_xlim(-0.1,0.1)
plt.plot(X_grid, pdf, label='Kernel')
plt.plot(X_grid,mlab.normpdf(X_grid, np.mean(ts5m.logreturns), np.std(ts5m.logreturns))) One should not simply compare a kernel density estimate with the theoretical normal distribution because the kernel density estimate is not an unbiased estimator of the true density. Hence, even if the true distribution of the random variable is the normal distribution, the kernel density estimate and the theoretical normal distribution will not be equal. Consequently, a comparison is only meaningful if you introduce an bias into the theoretical normal distribution or adjust your kernel density estimate for the bias.

• I agree. And it is an important point. I intend to use it as an indication and will include other analyses. Oct 29 '16 at 15:44