Hot answers tagged

26

Your ACF and PACF indicate that you at least have weekly seasonality, which is shown by the peaks at lags 7, 14, 21 and so forth. You may also have yearly seasonality, although it's not obvious from your time series. Your best bet, given potentially multiple seasonalities, may be a tbats model, which explicitly models multiple types of seasonality. Load ...


8

In the help page for ?tbats, we find that: The fitted model is designated TBATS(omega, p,q, phi, ,...,) where omega is the Box-Cox parameter and phi is the damping parameter; the error is modelled as an ARMA(p,q) process and m1,...,mJ list the seasonal periods used in the model and k1,...,kJ are the corresponding number ...


4

If you look at what tbats returns, you will see variance as one component. So fit <- tbats(x) rmse <- sqrt(fit$variance) will give you the RMSE. But even if you weren't sure what that component contained, you can always compute RMSE directly from the residuals of any model: res <- residuals(fit) rmse <- sqrt(mean(res^2)) You can also use the ...


4

The best way to decompose seasonal data using existing R packages is ceemdan() in Rlibeemd. This technique extracts seasonality of multiple periods. The defaults work well. It uses the Hilbert-Huang transform instead of the Fourier transform. The Fourier transform has a severe drawback in that it can only handle stationary, linear data when most series ...


4

They are the Box-Cox transformed values of the original data. I'll improve the docs in the next update.


3

The linear regression will be inefficient at best, and possibly inconsistent if the errors are non-stationary. A better approach is to add in multiple seasonal components as Fourier terms in the regression with ARIMA errors. An example with a single seasonal period is given at http://robjhyndman.com/hyndsight/forecasting-weekly-data/. That can be extended to ...


3

You have three years of daily data, which is a good amount. As you write, you may have multiple-seasonalities (intra-weekly and intra-yearly), so it is a good idea to look at models that can model such patterns, like bats or tbats. Let's start by looking at your series (I'm using R, and I created a data.frame similar to your hbf): with(hbf,plot(TheDate,...


3

tbats does not let you choose your own Box-Cox transformation parameter. The lambda argument will be ignored. You appear to have added the forecasts to the end of the data. So I will ignore the values after September 2016. Then you can force a transformation explicitly like this: x <- log(window(data_ts, end=c(2016,9))) fit <- tbats(x, use.box.cox=...


3

The questions you raise have been dealt with in R Time Series Forecasting: Questions regarding my output . Please look carefully at my detailed answer and all the comments in the discussion including those to the original question as I believe they are relevant to your problem. You might actually take the data that was provided in the post and use it as a ...


3

A TBATS model has an ARMA error structure. You are ignoring that when you simply subtract the level and seasonal terms. The residuals() function will extract the residuals properly, taking account of the ARMA error structure.


3

Your problem is not (so much) a lack of data. Your problem is that the data generating process changes abruptly multiple times. First there is a step change around Sep 20, then there is a period of strangely low variability at the beginning of November, almost two weeks of missing data at the beginning of December, and finally a precipitous drop at the end ...


2

No, that method is not valid in general. Here's a simple, illustrative counterexample. Assume that you have a random walk without drift: $$Y_t = Y_{t-1} + \varepsilon_t$$ $$\varepsilon_t \sim \mathcal{N}(0,1)$$ This process falls in the TBATS class (it is just an "ANN"-type ETS model with $\alpha=1$, without any complex seasonality, or Box-Cox transform, ...


2

Why would you avoid specifying the seasonal frequency of your data if you know it? It will be easier and likely more reliable then making some automated procedure estimate it for you. If you are not sure whether there is a seasonal pattern at a given frequency, you can try estimating a model with the seasonal component and another model without it. Then ...


2

Facebook has recently opened it R/Python API of 'Prophet Model'.It handles multiple level of seasonality and external event days can also be passed.Have a look- http://machinelearningstories.blogspot.in/2017/05/facebooks-phophet-model-for-forecasting.html https://facebookincubator.github.io/prophet/docs/forecasting_growth.html


2

Unfortunately I can't access your data set, but I think you can solve this problem by defining the start date when feeding tbats the time series. For instance: tbats_model <- tbats(ts(v01$Units, start=c(2016,1), f=12)) tbats_forecast <- forecast(tbats_model, h=1) tbats_forecast The c(2016,1) can be whatever date your data begins. Note: avoid useage ...


2

(T)BATS assumes seasonalities of constant length. If you have seasonality of varying length (often called "cycles" in the forecasting community), (T)BATS is probably not appropriate. Try using dummies of appropriate lengths. Of course, this presupposes that you can already forecast the length of the current (and later) cycles. If you are good at forecasting ...


2

Electricity prices are driven by external factors that depend on network effects and weather, especially if weather-dependent renewable energy is involved. Your negative prices indicate periods where someone generated too much electricity and needed to offload it - by paying someone else to take it. Things like this happen, e.g., when Germany generates more ...


2

First, you could read the paper that introduced the method: https://robjhyndman.com/publications/complex-seasonality/ But that might be hard going if you don't have a statistics background. If you just want to know what an ARMA model is, check out this chapter from my textbook: https://otexts.org/fpp2/arima.html To see an ARMA error in a model that is ...


2

Note the comparisons between the vertical axes of your decomposition plots. The seasonal signals are tiny compared to the other signals and the noise. This earlier thread is closely related: Even after seasonality adjustment, seasonality still remains. Why? Figuring out which seasonality is "real" is a bit of a philosophical question. As long as you don't ...


1

Any chance you would know if this approach sounds reasonable? It might work, but it would be overkill. TBATS was developed for complex seasonalities, for example daily sales data which exhibits both weekly and yearly seasonality. You have 2 years of monthly data - that's not granular enough to exhibit any complex seasonalities - at most you can detect a ...


1

freq=365.25/7 means you are assuming a year-long seasonality in your data. Since your results don't look good, try removing the seasonality by setting freq=1. You can also try using the ets and auto.arima functions with freq=1 to see what kind of results they give. If the seasonality of your data is not strong, using a non-seasonal model will likely give ...


1

Omega equals 1, this could imply that no Box-Cox transformation is necessary (see De Livera et al. 2010, S. 25-27 in footnote *). An additional note: Also consider the source code of the TBATS function by typing 'tbats' to the R console and hitting enter. Since I can't reproduce your problem, I just can guess, that your data set may contain negative values ...


1

The "random" component in a decomposition is the remainder from subtracting trend+season from y. So in the TBATS model, the residuals are the equivalent.


1

Looking at the source code of the package: aic <- likelihood+2*(length(param.vector$vect)+nrow(x.nought)) x.nought is some additional parameter particular to the tbats model, and I can't really comment on it. It becomes clear though the likelihood object is actually equal to $-2\ln(L)$.


1

You can use the function tbats.components on a tbats model fit. It will yield a matrix of time series containing the level, the slope and the season of the fitted model, as well as the actual observations. You can then simply subtract the components from the observations to obtain the residuals: > library(forecast) > fit <- tbats(USAccDeaths) > ...


1

Ponthu, I took your data and ran it in Autobox. Both of your events are important. Months 1,2,3 and 12 are higher than the rest of the months. Day 4 is typically 303 units higher than the other days of the week. You can simulate this by creating 11 dummy variables for the monthly effects, 6 dummies for the day of the week, etc. For Channel 2 if you ...


1

As @Aksakal wisely pointed out daily data analysis can reveal a ton of information. Look at http://www.autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/53-capabilities-presentation particularly slides 42-55 for a demonstration of this. One can break out daily-effects, weekly effects , monthly effects , level shifts . local trends in ...


1

You can look at time series cross-validation, but as you note, there is not overly much you can do with the results, since tbats() does not allow you to explicitly set parameters. You can, of course, determine based on your cross-validation whether or not to use a Box-Cox transformation or a trend etc. The help page ?tbats gives you a list of function ...


1

Why do most seasonal time series and forecasting algorithms presume that the seasonal period is (are) known? Here are a few reasons: For most applications, seasonal cycle lengths are known. Retail sales exhibit intra-yearly and intra-weekly seasonality. In Islamic countries, they exhibit a known irregular seasonality connected with the Islamic calendar. ...


Only top voted, non community-wiki answers of a minimum length are eligible