3
$\begingroup$

I want to use a Savitzky-Golay filter to smooth some data. There is a right width to use based on the data that it is smoothing. A number of papers basically use "eyeball norm" on the parameters but that feels like voodoo.

How do I get a measure like AIC to tell me which implementation is the proper one to use?

When I think about AIC, I do so in terms of RSS, number parameters, and number of degrees of freedom. I can measure RSS for the smoothed data, but I don't know how to think about parameter count or number of degrees of freedom in this context. Hints or pointers would be appreciated. Clear explanation would be delightful.

I'm personally using either R, or LabVIEW, but the actual software package isn't important.

Work so far:

  • This post says "k is 2 parameters, n is samples, use RSS" without providing references or derivation.
  • This reference is about the 'PoMoS' library which claimes to use genetic algorithms and information criteria to find optimal polynomial structure of time-series.

References:

$\endgroup$
  • 1
    $\begingroup$ It may be worth noting that the AIC is often viewed as a proxy for Kullback-Liebler divergence. $\endgroup$ – Mike Hunter Dec 6 '16 at 16:18
  • $\begingroup$ @DJohnson - AIC is derived from Kullback-Liebler, with assumptions. $\endgroup$ – EngrStudent Dec 6 '16 at 17:28
3
$\begingroup$

Economists have some rules of thumb they use ... but they don't call it Savitzky-Golay (who wrote their paper in 1964), they call it Hodrick-Prescott (1997). [Actuaries call it Whittaker-Henderson graduation (the usual reference for Whittaker being his 1923 book, and for Henderson, papers in 1916 and 1924), so they do somewhat better at naming it for the right people. But it's basically just discrete smoothing splines, where derivatives of some order are replaced with differences of some order.]

However, those rules that are used for Hodrick-Prescott filtering are based on some assumptions and approximations and don't necessarily compare well with smoothing parameters chosen in other ways.

One thing you might do to choose a smoothing parameter is some form of crossvalidation (if you have an implementation that deals with missingness); if your primary aim is to forecast you could substitute mean square one-step ahead prediction error.

However, it's not hard to estimate degrees of freedom consumed by the fit; taking the approach in Ye (1998) [1] of summing the partial derivatives of the fit with respect to the observations $\sum_i \frac{\partial \hat{y}_i}{\partial {y}_i}$ to obtain model d.f. (a definition which is consistent with our usual notions when we apply that to cases where our usual notions work).

In this particular case, this calculation is easily written as the trace of a matrix. If you can write $\hat{Y} = HY$ - which is easy in this case - then model df is the trace of $H$.

(You could argue for it from more basic considerations but this is a fairly intuitive way to look at it to my mind, and generalizes to a very wide array of methods for getting a some fit.)

[1] Ye, J. (1998),
"On measuring and correcting the effects of data mining and model selection."
J Am Statist Assoc., 93, pp120–31.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Your help with nomenclature, with good references, and with the point on how to determine degrees of freedom are quite helpful. Thank you. $\endgroup$ – EngrStudent Apr 7 '16 at 12:34
  • $\begingroup$ This leads to the Kalman filter. It is interesting that the Kalman filter and the AIC have a clean and direct bridge. How fun. $\endgroup$ – EngrStudent Apr 8 '16 at 11:07
1
$\begingroup$

I don't think you can really avoid some subjectivity here. The whole concept of smoothing is based on the idea that there is some slowly-varying signal that will be more apparent or meaningful if you remove variation around it, and bandwidth is the parameter that determines where in frequency terms the line is drawn between signal and noise. That means that the ideal bandwidth is not simply a property of the data, but also of what you want to achieve.

If you can make explicit your objective for smoothing it may help to determine a sensible bandwith. For example if you just want "to make the chart look smooth" then eyeballing a selection and picking the one you like best does exactly what you want! Or perhaps you could pick a function to maximise such as at association between the smoothed series and some other series, but while this adds an objective element to the process you have still chosen both the series and measure of association.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.