35
$\begingroup$

Is there a good way to measure smoothness of a time series in R? For example,

-1, -0.8, -0.6, -0.4, -0.2, 0, 0.2, 0.4, 0.6, 0.8, 1.0

is much smoother than

-1, 0.8, -0.6, 0.4, -0.2, 0, 0.2, -0.4, 0.6, -0.8, 1.0

although they have same mean and standard deviation. It would be cool if there is a function to give me a smooth score over a time series.

Improve this question
$\endgroup$
7
  • 6
    $\begingroup$ Smoothness has a well-defined meaning in the theory of stochastic processes. ("A variogram is a statistically-based, quantitative, description of a surface's roughness": goldensoftware.com/variogramTutorial.pdf, p. 16.) Smoothness is related to the extrapolation of the variogram to zero distance. (The SD of successive differences and the lag-one autocorrelation are quick and dirty versions of this). The essential information is contained in the coefficients of the Taylor series at 0. E.g., a non-zero constant is rough indeed; a high-order zero at 0 indicates a very smooth series. $\endgroup$
    – whuber ♦
    Mar 14, 2012 at 16:59
  • 1
    $\begingroup$ I've heard of hurst exponents too. $\endgroup$
    – Taylor
    Mar 14, 2012 at 18:22
  • 1
    $\begingroup$ @whuber: that's an answer, not a comment. $\endgroup$
    – naught101
    Aug 24, 2012 at 3:21
  • 2
    $\begingroup$ @naught101 I humbly beg to differ: my comment is apropos a related situation and it refers only to the theoretical process used to model spatial data, not to how one would actually estimate that smoothness. There is an art to that estimation with which I am familiar in multiple dimensions, but not in one, which is special (due to the direction of time's arrow), so I hesitate to claim that applying the multidimensional procedures to time series is at all conventional or even a good approach. $\endgroup$
    – whuber ♦
    Aug 24, 2012 at 3:24
  • 1
    $\begingroup$ @whiner: fair call. My understanding is limited, but a web search tells me that the a variogram in one dimension is a correlogram (or equivalent to one), which support something like cyan's answer. I don't really see how directionality impacts smoothness - surely a sawtooth wave is just as (un)smooth as a reverse sawtooth... $\endgroup$
    – naught101
    Aug 24, 2012 at 9:44

4 Answers 4

Reset to default
30
$\begingroup$

The standard deviation of the differences will give you a rough smoothness estimate:

x <- c(-1, -0.8, -0.6, -0.4, -0.2, 0, 0.2, 0.4, 0.6, 0.8, 1.0)
y <- c(-1, 0.8, -0.6, 0.4, -0.2, 0, 0.2, -0.4, 0.6, -0.8, 1.0)
sd(diff(x))
sd(diff(y))

Update: As Cyan points out, that gives you a scale-dependent measure. A similar scale-independent measure would use the coefficient of variation rather than standard deviation:

sd(diff(x))/abs(mean(diff(x)))
sd(diff(y))/abs(mean(diff(y)))

In both cases, small values correspond to smoother series.

Improve this answer
$\endgroup$
9
  • 2
    $\begingroup$ That score isn't scale-invariant, which may or may not make sense depending on the application. (And my own suggestion is scale-invariant, so the same concern applies to it.) Also, it's worth pointing out that for the above score, smaller values indicate smoother time series. $\endgroup$
    – Cyan
    Mar 14, 2012 at 4:02
  • 2
    $\begingroup$ Thanks @Cyan. I've now added a scale-independent version as well. $\endgroup$
    – Rob Hyndman
    Mar 14, 2012 at 5:14
  • 3
    $\begingroup$ Do you really intend to include diff in the denominators? The values would algebraically reduce to (x[n]-x[1])/(n-1) which is a (crude) measure of trend and ought, in many cases, to be extremely close to zero, resulting in an unstable and not terribly meaningful statistic. I'm puzzled by that, but maybe I'm overlooking something obvious... $\endgroup$
    – whuber ♦
    Jul 25, 2012 at 22:10
  • 1
    $\begingroup$ I used diff to avoid an assumption of stationarity. If it was defined with the denominator abs(mean(x)) then the scaling would only work when x was stationary. Taking diffs means it will work for difference stationary processes as well. Of course, diffs may not make x stationary and then there are still problems. Scaling time series is tricky for this reason. But I take your point about stability. I think to do anything better would require something substantially more sophisticated --- using a nonparametric smoother for example. $\endgroup$
    – Rob Hyndman
    Jul 25, 2012 at 22:53
  • 1
    $\begingroup$ I would have thought a constant trend should be perfectly smooth, so the answer should be 0. $\endgroup$
    – Rob Hyndman
    Mar 28, 2019 at 10:15
17
$\begingroup$

The lag-one autocorrelation will serve as a score and has a reasonably straightforward statistical interpretation too.

cor(x[-length(x)],x[-1])

Score interpretation:

  • scores near 1 imply a smoothly varying series
  • scores near 0 imply that there's no overall linear relationship between a data point and the following one (that is, plot(x[-length(x)],x[-1]) won't give a scatterplot with any apparent linearity)
  • scores near -1 suggest that the series is jagged in a particular way: if one point is above the mean, the next is likely to be below the mean by about the same amount, and vice versa.
Improve this answer
$\endgroup$
3
$\begingroup$

To estimate the roughness of an array, take the squared difference of the normalized differences, and divide by 4. This gives you scale-independence (because of the normalization), and ignores trends (because of using the second difference).

firstD = diff(x)
normFirstD = (firstD - mean(firstD)) / sd(firstD)
roughness = (diff(normFirstD) ** 2) / 4

Zero will be perfect smoothness, 1 is maximal roughness.

You then either use the sum of this measure, or its mean, depending on whether you want your roughness measure to be length-independent.

I think this may be the same as a previous answer elsewhere:

And similar things are discussed in academic sources like this and this, saying we should integrate the squared second derivative.

I don't read algebra, so I'm not sure if what I'm suggesting is quite the same as any of these.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Why is it divided by 4? $\endgroup$
    – Ajay Maity
    Sep 8, 2021 at 12:49
0
$\begingroup$

You could just check the correlation against the timestep number. That would be equivalent to taking the R² of a simple linear regression on the timeseries. Note, though, that those are two very different timeseries, so I don't know how well that works as a comparison.

Improve this answer
$\endgroup$
1
  • 5
    $\begingroup$ That would be a measure of the linearity with time, but not of the smoothness. $\endgroup$
    – Rob Hyndman
    Mar 14, 2012 at 5:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.