4
$\begingroup$

The nature of my problem combines seasonality (yearly repeated) and trend (over all data). I have one observation per month. To simplify, let's say I have this: enter image description here

Which can be perceived as this: enter image description here

So far, I observed that trying any type of linear regression (also including support vector regression) on all data does not give me any acceptable results.

So I changed to a different approach:

  1. Learning seasonality using entire years as units to train (here I learn the curvy shape in one year)
  2. Learning the trend by simple linear regression on all data

I combine results from 1 and 2 in order to magnify shape learned in 1 by trend increase in 2. So far, it gives me acceptable results, but my humble intuition (my experience is mostly related to classification problems) is that there must be a better approach which is specifically suited for this kind of problem.

Actually I have two questions related to the problem:

  1. Is there a better approach?

  2. Let's say I want to predict a whole year, what's best: to include predicted data to predict further months in a more informed way OR to avoid it and predict strictly based on training data (taking into account that predicted data might be missleading)?

$\endgroup$
5
+50
$\begingroup$

There are several methods and models for this kind of analysis, for example: exponential smoothing, ARIMA time series models or structural time series models. The topic is too broad to be covered here. Below, I give some examples in R just for illustration. For further details, you may start for example looking at this online textbook (Forecasting: principles and practice by R. Hyndman and G. Athana­sopou­los).

As regards your second question, my recommendation is in line with Winks's answer. I would stick to fit a model using the sample data and then apply one-step ahead forecasts. If any, you can remove the last year from the sample and get forecasts for that year upon different methods or models. Then, you can choose the model with the best performance according to some accuracy measure such as the Mean Absolute Error (minimum MAE).


The monthly totals of airline passengers (1949 to 1960) is a common example used to illustrate time series models and methods (this series has also some resemblance to the patterns that you show).

In R, you can obtain one-year-ahead forecasts (blue line) by means of the Holt and Winters filter as follows:

x <- log(AirPassengers)
res1 <- HoltWinters(x)
pred1 <- predict(res1, 12)
plot(cbind(x, pred1), plot.type = "single", ylab = "", col = c("black", "blue"))

Hold and Winters

Obtaining confidence intervals for the forecasts is easier by means of parametric methods. An ARIMA model can be used to obtain forecasts and 95% confidence intervals (red dotted lines) as follows:

require("forecast")
res2 <- auto.arima(x)
pred2 <- predict(res2, 12)
plot(cbind(x, pred2$pred, pred2$pred + 1.96 * pred2$se, pred2$pred - 1.96 * pred2$se), 
  plot.type = "single", ylab = "", col = c("black", "blue", "red", "red"), lty = c(1,1,2,2))

ARIMA

The basic structural time series model is suited for the kind of components that you describe. In addition to forecasts and confidence intervals, an estimate of the trend and seasonal components is also obtained (this post gives an example that includes forecasts for the components):

require("stsm")
mod <- stsm.model(model = "BSM", y = x, transPars = "StructTS")
res3 <- maxlik.td.optim(mod, KF.version = "KFKSDS", 
  KF.args = list(P0cov = TRUE), method = "L-BFGS-B", gr = "numerical")
pred3 <- predict(char2numeric(res3$model), x, n.ahead = 12)
plot(cbind(x, pred3$pred, pred3$pred + 1.96 * pred3$se, pred3$pred - 1.96 * pred3$se), 
  plot.type = "single", ylab = "", col = c("black", "blue", "red", "red"), lty = c(1,1,2,2))

basic structural time series model

$\endgroup$
4
$\begingroup$

If your data are simple, and you are just toying around, the decomposition into seasonal and overall trend should already be pretty good. But if you want to dig deeper, there is a more formal approach: Using Kernels to define the relations between your points.

The best exemple I have seen of this kind of task is on the Mauna Loa CO2 concentration dataset (Wikimedia Commons illustration)

Mauna Loa CO2 Concentration dataset: visualization of year average and monthly deviation from average.

A possible model for this dataset is shown in Gaussian Processes for Machine Learning, by Rasmussen & Williams (Second edition available online), in section 5.4.3.

Warning: Gaussian Processes (Wikipedia) are not easy to understand, especially if you do not have a background in Kernels. In a very simplified way, it is like doing Nearest Neighbour regression (a point has the value of close points), but using Kernels (Wikipedia) to encode prior knowledge about the data and the form of the function. What kernels allow you to do is specify how "related" two points are.

In this case, you can build a kernel that defines "relatedness" by how close in month we are, and how close in years, such that 12/2015 is related to 01/2016 and 12/2014, but not so much related to 06/2015 and not to 06/2004.

Since you mention Support Vector Regression, you could probably do something similar using such a Kernel there.

Considering your second question, it would be best to leave predicted data outside of your training data. Doing this does not add information, and if your model is biased, the predicted data will only reinforce this bias and lead to poorer prediction. This is true for all methods.

$\endgroup$
  • $\begingroup$ just want to make clear that your answer is perfectly valid and useful but I could only give the bounty to one user. I finally gave it to @javlacalle because its code examples were fully aligned with my problem. Sorry and thank you very much! $\endgroup$ – Guiem Bosch Apr 4 '16 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.