# Time series modeling with high-frequency data

I'm looking for some forecasting advice when dealing with seasonal time series data that has a large number of observations. By "large" I only mean a few thousand --- I'm used to such sizes in Data Mining being considered pretty small, but it seems that in time series modeling that's pretty unwieldy for many of the tools I've tried.

For example, here's a toy data set that records an observation once per minute, for five days:

set.seed(123)
t <- 1:(5*24*60)
x <- ts(15 + 0.001*t + 10*sin(2*pi*t/(length(t)/5)) + rnorm(length(t)), freq=length(t)/5)
plot(x, type='l') (In my real operational data set, the values are observed at irregular intervals, but I've regularized them by doing something like x <- approx(d$t, d$x, xout=1:(5*24*60)) first. Advice on whether that's advisable, or alternative approaches, is welcome too.)

So the seasonality in this data set has a lag of 1,440 observations, which seems to be way outside the range that things like auto.arima() (in the forecast package) will find:

m1 <- auto.arima(x)
plot(forecast(m1)) And I'm not quite sure how to interpret the ets() function here, but it doesn't seem to be able to handle this size data, and it didn't seem to pick up on the seasonality:

> m2 <- ets(x, 'MAZ')
> plot(forecast(m2))
Error in forecast.ets(m2) : Forecast horizon out of bounds
> m2\$method
 "ETS(M,A,N)"


Where to go from here? Any suggestions?

## 1 Answer

ARIMA and ETS models are designed for relatively short seasonality (e.g., monthly or quarterly) and do not work well for seasonal periods that are much longer. The ets() error should be captured though --- I'll fix it in the next version of the forecast package.

You might try instead a Fourier series model:

library(forecast)
X <- fourier(x, 3)
m3 <- tslm(x ~ X + trend)
plot(forecast(m3,newdata=data.frame(X=I(fourierf(x,3,2880))),h=2880))


See my blog post on forecasting with long seasonal periods for more discussion.