7
$\begingroup$

I'm currently writing my bioinformatics thesis. One part of what I'm doing involves performing a KDE on univariate data. I'm using R, and in the function documentation (for the density function) it says the Sheather-Jones bandwidth selection method is generally recommended. After trying a few different methods, I did notice that SJ gave the best results. I also saw online (including here on CrossValidated) that people usually say it's the recommended method.

Upon reviewing my thesis, one of my committee members asked me to briefly describe the method. I honestly have ZERO idea what it actually does. I tried reading their paper, but unfortunately both the abstract and the intro didn't really have any kind of simple lay person's explanation of what their method actually does. I could not find much information online. I'm not one to easily "outsource" my problems to the online community, I usually try to figure out everything on my own, but this seems to be way out of my league and I really need some help here.

Does anyone have a very brief 1-2 sentence explanation of how SJ bandwidth selection works or why it's regarded as the most popular? Thanks

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ When you say "the reference" please be explicit about which particular one you mean -- i.e. give a complete reference (many authors publish more than one thing together on a given topic, it's possible for people to end up explaining a reference you don't intend). I don't think you should expect a 1-2 sentence explanation that's much use. A paragraph, perhaps. As for "regarded as the most popular", you'd have to justify the claim -- regarded by whom? Is it most popular? Perhaps, but I don't know how to tell for sure. $\endgroup$
    – Glen_b
    Apr 4, 2016 at 0:56
  • 1
    $\begingroup$ Do you mean this one? S. J. Sheather, and M. C. Jones. (1991) "A Reliable Data-Based Bandwidth Selection Method for Kernel Density Estimation." Journal of the Royal Statistical Society. Series B, 53 (3) pp683-690. $\endgroup$
    – Glen_b
    Apr 4, 2016 at 1:02
  • $\begingroup$ I've now included a link to another CV question relating to AMISE into my answer that may help; Rob H's answer there specifically refers to the Sheather-Jones bandwidth estimator. $\endgroup$
    – Glen_b
    Apr 4, 2016 at 3:45

1 Answer 1

Reset to default
5
$\begingroup$

Assuming it is indeed the preferred bandwidth estimator in Sheather and Jones' (1991) JRSS-B paper [1] that you mean (specifically, $\hat{h}_{2S}$), here's a brief discussion (as requested), but a brief discussion of a highly technical topic is necessarily a little vague and cryptic.

The basic issue of finding efficient$^\dagger$ bandwidth estimators boils down to finding a good estimate of $R(f'')$ (where $R(g) = \int g^2(x) dx$), the integrated squared second derivative of the density to be estimated -- i.e. the asymptotically optimal bandwidth depends on the second derivative of the very thing we wish to estimate!

$^\dagger$ here, specifically in the sense of minimum asymptotic mean integrated squared error (AMISE) ... about which, see here

Why does the integrated squared second derivative matter? In effect it measures how "wiggly" the curve is over the range you're looking at. If you have a very wiggly curve you won't get a good estimate of it with a wide bandwidth because you'll average over a bunch of wiggles instead of following them. If you have a curve that's pretty straight it makes sense to have a much wider bandwidth (since you can reduce the noise in your estimate by including more data).

A number of bandwidth estimators use (in turn) a kernel based estimate of $R(f'')$.

Sheather and Jones include a bias term in their estimate of $R(f'')$ that had previously been neglected. This results in estimating $R(f''')$ (a lot of detail is being glossed over here).

How to summarize all that? It's an improved version of a kernel-based estimate of the optimal bandwidth, not that this is likely to help much.

As to why it's popular (I won't engage in idle discussion of whether it's the most popular, since it seems impossible to reliably assess), the abstract gives a highly plausible reason:

reliably good performance for smooth densities in simulations [...] second to none in the existing literature

i.e. it (demonstrably) works well in practice for a reasonably broad set of of cases.

[There have been - unsurprisingly - further suggested improvements in the last quarter century, but this bandwidth estimator remains popular.]

[1] S. J. Sheather, and M. C. Jones. (1991)
"A Reliable Data-Based Bandwidth Selection Method for Kernel Density Estimation."
Journal of the Royal Statistical Society. Series B, 53 (3) pp683-690

Improve this answer
$\endgroup$
7
  • $\begingroup$ This is indeed the method I was referring to. Thank you for the thorough explanation, I appreciate it, but unfortunately that's still way over my head as someone who doesn't have a statistics background. I'm not sure what MISE/AMISE are, but I do sort of understand "adding a bias term", so I suppose that's something :) $\endgroup$
    – DeanAttali
    Apr 4, 2016 at 3:19
  • 1
    $\begingroup$ @daattali I gave the complete term that AMISE is short for in my answer - it's the asymptotic mean integrated squared error. I could pretty much explain what that is in a few sentences if you want, though it could arguably be another question. If you want a full understanding of all the details of what the Sheather Jones estimator is doing, though, you'll need not a sentence, nor a paragraph, nor even a short paper ... to properly understand it all requires (at least) a book, possibly several. $\endgroup$
    – Glen_b
    Apr 4, 2016 at 3:24
  • $\begingroup$ @daattali Mean integrated squared error (MISE) is defined here. Presumably it's clear why this is a desirable thing to minimize. $\endgroup$
    – Glen_b
    Apr 4, 2016 at 3:36
  • $\begingroup$ I saw that Wiki page, thank you, but still could not understand intuitively what it means, and I was unable to find an English explanation anywhere for either MISE or AMISE. After reading your answer over a dozen times, I think AMISE is somehow related to the integral of the second derivative of $f$. Though I thought the integral of a derivative would result in the original value, so it seems a bit weird to me. I think, as you said, this might not be easily explained to a lay person. $\endgroup$
    – DeanAttali
    Apr 4, 2016 at 3:41
  • $\begingroup$ Your answer is great and attempts to explain it in simple terms, maybe I just need to read it and the Wiki page for MISE a few more times and understand the other few terms and then it'll click $\endgroup$
    – DeanAttali
    Apr 4, 2016 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.