3
$\begingroup$

I have an hourly time series of temperature:

(can be downloaded here: http://workupload.com/file/eFFPWvL)

plot(my_data$time, my_data$temperature, type = "l")

enter image description here

for which I would like to calculate a sine cosine function of the form:

enter image description here

As I understand the sine cosine are able to capture any complex form of seasonality/cyclicality. In my data, there is yearly seasonaly as well as hourly which I would like to capture.

I tried (in R) what I thought might be appropriate using the model estimation formula like:

fit <- lm(temperature~ time + sin(2*pi/365*time)+cos(2*pi/365*time) +
                sin(2*pi/(365*24)*time)+cos(2*pi/(365*24)*time),data=df)
lines(fit$fitted.values, col = 2)

But the result does not look promising.

enter image description here

Fist of all, the high frequency pattern does not seem to capture the within day fluctuations as there should be much more "waves" (for each day). Secondly, the magnitude of my fitted line is much lower than that of the data.

Can someone pelase tell me what am I doing wrong?

|cite|improve this question|||||
$\endgroup$
  • $\begingroup$ The way you've specified your model implies that every component trend will occur the same way: there's one annual trend, one hourly trend, and so on, and all of these manifest in the same way throughout the time series. Maybe this is sufficient as an approximation. But if not, you'll need to describe more clearly what your interest is, and how the current model is unsatisfactory. But in general, if you add more basis functions, your fit will improve (at risk of overfitting). $\endgroup$ – Sycorax says Reinstate Monica Dec 12 '16 at 1:18
  • $\begingroup$ @Sycorax thanks! Dont you think that my high frequency pattern completrly misses the daily frequency? I have data of a year and there should be 365 waves? And the amplitude seems to me too flat... $\endgroup$ – DatamineR Dec 12 '16 at 1:28
  • $\begingroup$ Maybe you should add some terms of intermediate timescale; monthly, weekly and so on. In extrema, you could make the regression hit every observation if you just used an indicator for each timestamp. Or you could use a much larger family of basis functions and impose regularization on the high-frequency terms. $\endgroup$ – Sycorax says Reinstate Monica Dec 12 '16 at 1:42
  • 1
    $\begingroup$ To qualitatively check your data for "daily cyclicity", you might consider compositing all the data into a "1 day" plot. That is, plot $(x,y)$ points with $x=t-t_0$ and $y=T-\bar{T}$, where for each day $t_0$ is midnight and $\bar{T}$ is the average temperature. You could then do simple visualization by e.g. looking at $y$ percentiles over each hourly $x$ bin, to see if the median $y_{50}$ has any regular cyclicity. And if so, if it has a simple sinusoidal form. $\endgroup$ – GeoMatt22 Dec 12 '16 at 3:03
2
$\begingroup$

Instead of 

fit <- lm(temperature~ time + sin(2*pi/365*time)+cos(2*pi/365*time) +
                sin(2*pi/(365*24)*time)+cos(2*pi/(365*24)*time),data=df)
lines(fit$fitted.values, col = 2)

try 

fit <- lm(temperature~ time + sin(2*pi/(365*24)*time)+cos(2*pi/(365*24)*time),data=df)
lines(fit$fitted.values, col = 2)

You said your data are hourly. If this is true, you cannot estimate an hourly trend (https://en.wikipedia.org/wiki/Nyquist_frequency). This regression I suggest will fit a regression onto a yearly frequency (1 cycle for every 365*24 hours). I got rid of the frequency that corresponded to 1 cycle for every 365 hours, which probably isn't special. As long as you understand the units of frequency, and as long as you keep them between $0$ and a half (not inclusive), you should be alright.

Also, you are including time as a regressor in your code, but not in your math. You may or may not want to remove the time from the above call to lm().

If you want to throw in a few more frequencies, use a periodogram (https://stat.ethz.ch/R-manual/R-devel/library/stats/html/spec.pgram.html) to tell you which frequencies are pertinent.

|cite|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.