Suppose we have toy daily temperate data and we want to fit a model.

enter image description here

A reasonable thing to do is fitting a periodic model with Fourier basis

$$ f(x)=\beta_0+\beta_1 \cos(2\pi x/24)+\beta_2 \sin(2\pi x/24) $$

So the Fourier basis expansion of data matrix $\mathbf X$ is

$$ \begin{bmatrix} 1&\cos 0 & \sin 0 \\ 1&\cos \frac \pi 4 & \sin \frac \pi 4 \\ \cdots & \cdots & \cdots\\ 1&\cos \frac {7\pi} 4 &\sin \frac {7\pi} 4 \end{bmatrix} $$

On the other hand, suppose I do not know Fourier expansion and only know polynomial fit. So I fit data with a third order polynomial, where

$$ f(x)=\beta_0+\beta_1 x+\beta_2 x^2 +\beta_3 x^3 $$

The polynomial basis expansion of data matrix $\mathbf X$ is (for demo purpose, I am not using orthogonal polynomial which will be ill conditioned in real world problems.)

$$ \begin{bmatrix} 1&\ 0 & 0 & 0 \\ 1& 3 & 3^2 & 3^3\\ & \cdots\\ 1&\ 21 &21^2 & 21^3\end{bmatrix} $$

The two fits are shown below, and they are "similar". My question is, what's wrong with the polynomial fit on periodical data? In this case we do not have the extrapolation and the time should always be $[0,23]$

    d = data.frame(t=c(0,3,6,9,12,15,18,21),
    coeff = solve(t(X) %*% X, t(X) %*% d$temp)
    X2=cbind(1,d$t, d$t^2, d$t^3)
    coeff_2 = solve(t(X2) %*% X2, t(X2) %*% d$temp)

    d_new = seq(0,24,0.1)
    X2=cbind(1,d_new, d_new^2, d_new^3)

    lines(d_new,X %*% coeff, type='l',col='red')
    lines(d_new,X2 %*% coeff_2, type='l', col='blue')

enter image description here

  • 3
    $\begingroup$ Do the discontinuities at t = 24/0 for the polynomial fit annoy you? $\endgroup$ – Cliff AB Jul 21 '16 at 20:00
  • 1
    $\begingroup$ Why not just use a gaussian process regression with a periodic kernel? $\endgroup$ – Sycorax Jul 21 '16 at 20:02
  • 1
    $\begingroup$ For these purposes I use a periodic version of splines, often quadratic or cubic. $\endgroup$ – whuber Jul 21 '16 at 20:24
  • 1
    $\begingroup$ @whuber that's interesting, can you please show the equations or point to a relevant R package? $\endgroup$ – DeltaIV Jul 22 '16 at 8:34
  • 2
    $\begingroup$ yes @whuber could you share the equations for periodic version of splines? $\endgroup$ – Haitao Du Jul 22 '16 at 12:56

In just the dataset you've provided, the only real downside to using polynomials over the Fourier basis is the issue of discontinuity at $T = 0$ and $T = 24$. As you stated, you can add constraints to fix this up if you really wished to.

But more typically for this type of data, we observe several cycles. In this case, it would be the number of days of data. The whole point is to take advantage of the fact that 3pm on Monday has very similar features to 3pm on Tuesday. This relation would not show up at all in the "vanilla" polynomial expansion, and so you would not be borrowing at all from different cycles for estimation. For similar reasons, you would have almost no hope of getting a good extrapolation, even just 1 day out, where as even from a very basic Fourier expansion, you could say "I think at 3pm tomorrow, it will probably be the same heat as it usually is at 3pm".

| cite | improve this answer | |
  • $\begingroup$ Suppose, we have 10 days of data, each day we have 8 records (as shown in the toy data). You are saying, we should treat 1 time series with 80 data points? I originally think we should "aggregate" all 10 days data and fit a polynomial. Where for example, at time 1, there are 10 data points. $\endgroup$ – Haitao Du Jul 25 '16 at 15:51
  • 1
    $\begingroup$ @hxd1011: if you consider what happens in the Fourier transformation, that's actually kind of what's happening automatically, as $cos(2\pi) = cos(0)$. $\endgroup$ – Cliff AB Jul 25 '16 at 16:17

The wrong is that to exactly capture the simplest periodic process such as a monochrome sine wave you need infinite number of polynomial terms. Look at Taylor expansion formula.

Intuitively you want to fit function that (in some sense) looks like your underlying process. This way you'll have the fewest number of parameters to estimate. Say you have a round hole, and need to fit a cork into it. If your cork is square it's harder to fit it well than if the cork were round.

| cite | improve this answer | |

Discontinuity at $T=0$ and $T=24$ is problem. In fact, the plot is misleading because it only plots $T$ up to $21$. If we change the plot code as:


We can see 3rd order polynomial is not a good fit:

At time $0$, the temperature is $-1.7$, but next day at time $0$ the temperature at $-7.04$ !:

enter image description here

In addition, it is very nature to have function input $T$ as any real number, instead of limited to 0 to 23.

For example, when $T=25$ it just means 1:00 in next day and $T=-1$ means 23:00 in previous day. Using polynomial basis we need to make to inside 0 to 23 to generate output.

But with Fourier basis expansion, everything is build in.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.