1
$\begingroup$

Below I am showing the kernel density with the size of the informal economy, and would appreciate support on interpreting this. For instance, what does the of the Kdensity line around .017 represent relative to the normal density line?

What does a bandwidth of 7.31 tell us?

enter image description here

$\endgroup$
3
  • $\begingroup$ Contrary to your description the graph is clearly showing the distribution of residuals from some earlier model. Although the program does what it is designed to do, many practitioners regard a normal quantile plot as a better check on normality of residuals (which is often desirable but not essential). $\endgroup$
    – Nick Cox
    Apr 1 '21 at 21:15
  • $\begingroup$ The density is subject to the rule that the area under the curve must total $1$, as it represents the total probability. This is easiest to think about by imagining replacing the density by a rectangle with the same area. The base of the rectangle is the range from (roughly) $-50$ to $50$, so about $100$, So the height of the rectangle must be about $0.01$, i.e. $\sim 100 \times \sim 0.01 \approx 1$. That seems about right for the average density. $\endgroup$
    – Nick Cox
    Apr 1 '21 at 21:28
  • $\begingroup$ The units of measurement of the residuals are the same as those of your response or outcome variable. The units of density are the reciprocal of those units. The units therefore wash out of the calculation. $\endgroup$
    – Nick Cox
    Apr 1 '21 at 21:29
1
$\begingroup$

Given a random sample from a population, a kernel density estimator (KDE) seeks to estimate the density function of the population distribution. You can read Wikipedia's article on KDEs or various other Internet pages for details of how a KDE is formed. (I have found referenced papers by Silverman to be extraordinarily clear.)

Roughly speaking, one chooses the shape of a 'kernel' density (often normal, sometimes uniform or others) and then makes a mixture of several such distributions as the KDE. The smaller the bandwidth, the more the components of the mixture. Results are often smoother than you get by trying to estimate a density function using a histogram. You can think of a KDE as a 'smoothed histogram', but the KDE works entirely independently of the histogram.

If you have a large sample, you will generally get a KDE that comes closer to the density function of the population.

Suppose you have a sample of size $n = 500$ from $\mathsf{Gamma}(\mathsf{shape}=5,\mathsf{rate}=0.1),$ which has $\mu=50,\sigma^2=500.$

set.seed(2021)
x = rgamma(500, 5, 0.1)
summary(x);  sd(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   4.33   32.18   44.91   49.11   62.64  163.26 
[1] 23.9333   # sample SD

Here is a histogram of the sample, a graph of the density function of $\mathsf{Gamma}(5, .1)$ [dotted black], individual observations [tick marks], and the default KDE from R [solid brown].

hdr = "n = 500: Sample from GAMMA(5,.1) with Density (dotted) and KDE"
hist(x, prob=T, col="skyblue2", br=20,  main=hdr);  rug(x)
 curve(dgamma(x, 5, .1), add=T, lwd=2, lty="dotted")
 lines(density(x), lwd=2, col="brown")

enter image description here

With obvious changes in the R code, here is a similar plot with $n = 10\,000$ observations. Here we have used KDEs with bandwidths half (with parameter 'adj=.5' in 'density') and double the default size.

set.seed(401)
x = rgamma(10^4, 5, .1)
hdr = "n = 100,000: Sample from GAMMA(5,.1) with KDEs of two bandwidths"
hist(x, prob=T, col="skyblue2", br=20,  main=hdr)
 curve(dgamma(x, 5, .1), add=T, lwd=2, lty="dotted")
 lines(density(x, adj=.5), lwd=2, col="green3")
 lines(density(x, adj=2), lwd=2, col="red", lty="dashed")

enter image description here

The narrower bandwidth (green) is not smooth near the mode, the wider bandwidth (red) is not quite right in the lower tail. Either KDE is better than the histogram with about 20 bins. In my experience, the default bandwidth in R is about right. (Default gaussian kernels are used throughout.)

R does not report the exact bandwidth it uses. I am happy to consider the bandwidth as as a technical matter and have found it more useful to see how well the KDE matches a histogram (or the true density curve, if known).

$\endgroup$
5
  • 1
    $\begingroup$ I disagree mildly. It's very important to realise that bandwidth choices -- whether yours or made by the code -- are -- even if well chosen -- still arbitrary and that what you see or do not see is dependent on bandwidth and that there should be some exploration of sensitivity to bandwidth choices. Kernel shape shouldn't matter so much, for the same degree of smoothing. $\endgroup$
    – Nick Cox
    Apr 1 '21 at 21:06
  • 1
    $\begingroup$ Also, although details vary, in principle the kernel density estimate from $n$ observations is the sum of just as many miniature distributions each based on a sample value. So, what you call the number of components doesn't depend on the bandwidth at all. What does depend on the bandwidth is how much they are smeared out and therefore how lumpy or smooth the estimated density is. Some algorithms use very good approximations to that recipe. $\endgroup$
    – Nick Cox
    Apr 1 '21 at 21:07
  • 1
    $\begingroup$ Your example shows a common pitfall often ignored in practice. The gamma you generate from isn't defined for negative values but the kernel procedure does not know that and happily smooths the distribution into an impossible zone. There are fixes for this. A similar problem can easily apply if values are close to or equal to any upper bound. $\endgroup$
    – Nick Cox
    Apr 1 '21 at 21:11
  • 1
    $\begingroup$ If I'm in doubt, I usually try doubling and halving the bandwidth to see if that makes much difference--as shown in my answer. But wouldn't know how to interpret the exact number in a constructive way. // As implemented in R, KDE's don't do well near the boundaries of support unless the density is low there. Not sure how to handle that. $\endgroup$
    – BruceET
    Apr 1 '21 at 21:13
  • 1
    $\begingroup$ It's the closing statement "I am happy to consider the bandwidth as as a technical matter" that undermines your careful analysis slightly. People using this (FWIW I am a geographer with no formal mathematical education beyond age 17) have little difficulty grasping that if you vary the degree of smoothing, the results will vary, on a par with the demonstration that should be in every introductory course that histograms depend on bin width. (The easiest fix for the boundary problem here is to estimate a density on the log scale and back transform, but no fix is perfect.) $\endgroup$
    – Nick Cox
    Apr 1 '21 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.