0
$\begingroup$

I am trying to compare a Kernel Density estimation with a normal distribution.

I use the following code to estimate the Kernel Density using a Epanechnikov kernel:

from sklearn.neighbors.kde import KernelDensity

X_grid = np.linspace(-5, 5, num=1000)

def silverman_bw(ts):
    return 1.3643*1.7188*len(ts)**(-0.2)*min(np.std(ts), np.subtract(*np.percentile(ts, [75, 25])))

kde = KernelDensity(kernel='epanechnikov', bandwidth=silverman_bw(ts5m.logreturns)).fit(ts5m.logreturns.reshape(-1,1))
pdf = np.exp(kde5m.score_samples(X_grid.reshape(-1,1)))

and the following code to generate the Normal Distribution I want to benchmark it up against

from scipy.stats.distributions import norm

normpdf = norm.pdf(X_grid)

The problem is obvious when I plot these two in the same plot; it is impossible to compare the two distributions due to different y-scales. The red line is the Kernel Density estimate and the blue-ish the Normal Distribution.

Kernel Density estimate and Normal Distribution plotted

What do I do in order to be able to compare these two? I am probably missing some scaling/standardization-something, but I just can't remember the precise theory, and I am unable to find answers in my academic litterature.

Improve this question
$\endgroup$
1
  • $\begingroup$ You could try plotting their ratio. $\endgroup$
    – user1566
    Commented Oct 28, 2016 at 20:28

3 Answers 3

Reset to default
1
$\begingroup$

The plot wouldn't look very different if you compared standard normal distribution with normal distribution parametrized by standard deviation of $\approx 0.15$...

Normal distribution with different sd's

Comparing two distributions that have standard deviations differing by a magnitude of over sixty times does not make much sense. Even if they did not differ in height of density (what is impossible), then either the distribution with greater standard deviation would be outside of your plotting area, or the one with smaller standard deviation would be just a thin line in the middle of the plot...

Normal distribution is a member of location-scale family, if $f(x)$ is a standard distribution with location $\mu=0$ and scale $\sigma=1$, then if you change it's location to some other $\mu'$ and scale to $\sigma'$, it becomes $f((x-\mu)/\sigma)/\sigma$. This means that if standard normal distribution at it's highest point $x=0$ has height of $0.3989423$, then when you change it's scale to $\sigma'$, it will get $\tfrac{1}{\sigma'}$ times higher.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for your answer Tim. You are certainly right. I was actually under the impression that the norm.pdf(x)-function scaled the pdf using x. I've tried scaling up the normpdf manually instead by multiplying with the std.dev of logreturns and adding the mean. It does not make the distributions comparable, unfortunately. $\endgroup$
    – mfvas
    Commented Oct 29, 2016 at 7:10
  • $\begingroup$ @mfvas norm.pdf lets you to provide location and scale as function parameters, moreover if $f(x)$ is a standard distribution, then when it is parameterized by scale you have to take $f(x/s)/s$. The same says it's documentation: "Specifically, norm.pdf(x, loc, scale) is identically equivalent to norm.pdf(y) / scale with y = (x - loc) / scale." $\endgroup$
    – Tim
    Commented Oct 29, 2016 at 7:29
  • $\begingroup$ Thanks Tim. It looks like i misunderstood how the function worked. Using the built-in normal distribution generator in matplotlib I now have a comparable plot. $\endgroup$
    – mfvas
    Commented Oct 29, 2016 at 8:23
  • $\begingroup$ @mfvas I'm glad it helped. I added some comment to make it more clear why did you observed what you did. $\endgroup$
    – Tim
    Commented Oct 29, 2016 at 11:13
0
$\begingroup$

I found a solution. It turns out, with the guidance from Tims answer, that the norm.pdf() function does not work like functions I normally work with in different programming languages and thus I needed to scale my variables differently.

Using the function from matplotlib.mlb instead, which works like I expected it to, I now have comparable graphs:

import matplotlib.mlab as mlab

def silverman_bw(ts):
    return 1.3643*1.7188*len(ts)**(-0.2)*min(np.std(ts), np.subtract(*np.percentile(ts, [75, 25])))

kde = KernelDensity(kernel='epanechnikov', bandwidth=silverman_bw(ts5m.logreturns)).fit(ts5m.logreturns.reshape(-1,1))
pdf = np.exp(kde5m.score_samples(X_grid.reshape(-1,1)))

f, ax = plt.subplots()
ax.set_xlim(-0.1,0.1)
plt.plot(X_grid, pdf, label='Kernel')
plt.plot(X_grid,mlab.normpdf(X_grid, np.mean(ts5m.logreturns), np.std(ts5m.logreturns)))

enter image description here

Improve this answer
$\endgroup$
0
$\begingroup$

One should not simply compare a kernel density estimate with the theoretical normal distribution because the kernel density estimate is not an unbiased estimator of the true density. Hence, even if the true distribution of the random variable is the normal distribution, the kernel density estimate and the theoretical normal distribution will not be equal. Consequently, a comparison is only meaningful if you introduce an bias into the theoretical normal distribution or adjust your kernel density estimate for the bias.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I agree. And it is an important point. I intend to use it as an indication and will include other analyses. $\endgroup$
    – mfvas
    Commented Oct 29, 2016 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.