Let's suppose I have a time series data that I would like to regress $y$ on $x$ and $Time$. See below for the dataset.

y   x   time
12  100 1
14  101 2
16  102 3
18  103 4
20  201 1
22  202 2
24  203 3
26  204 4

Approach 1:

One approach is to do a multiple linear regression or neural networks or SVM directly on the dataset above by treating time as an ordinal (?) or continuous variable. I can do a time series regression as follows:

$$y = a+\beta_1x+\beta_2 time+\varepsilon$$

where $\varepsilon$ is modeled as ARMA

Approach 2:

Alternatively I can rearrange data and create dummy variables for time as follows:

y   x   Time_1  Time_2  Time_3
12  100 1   0   0
14  101 0   1   0
16  102 0   0   1
18  103 0   0   0
20  201 1   0   0
22  202 0   1   0
24  203 0   0   1
26  204 0   0   0

and do a time series regression/neural network/SVM. For instance a time series would be

$$y = a+\beta_1x+\beta_2 time_1+\beta_3 time_2+\beta_4 time_3+\varepsilon$$

where $\varepsilon$ is modeled as ARMA.

Below are my questions:

  1. What is the right approach - 1 or 2?
  2. If we use the data mining approach such as a neural network or svm does it matter if we use either approach 1 or 2 ?
  3. Is approach 1 more parsimonious since we have 1 variable representing time as opposed to approach 2 which has 3 variables?
  • $\begingroup$ Out of curiosity, how do you model ARMA errors with NN or SVM? $\endgroup$ – B_Miner Feb 2 '14 at 17:02
  • $\begingroup$ @B_Miner I was referencing ARMA for errors in time series regression not for NN or SVM. $\endgroup$ – forecaster Feb 2 '14 at 17:45
  1. Modeling time continuously introduces the assumption that there is a linear influence of time upon the outcome, conditional upon $x$. However, adjusting for time as a fixed and random effect makes this interpretation a bit untenable.

  2. Yes it does matter, it matters in absolutely all scenarios. You can verify this by simulating data according to either linear model. When you fit categorical effects for linear time, you still consistently estimate the linear trend in time, but you "spend more" with regards to the degrees of freedom.

  3. In general, yes. There are fewer effects in the first model. However, the overarching idea of which model (categorical effects versus linear time) is correct can be most correctly addressed by asking: What is the scientific question?

| cite | improve this answer | |

What makes you think that time has any effect on the dependent variable?

I'd suggest plotting the dependent variable against time to gauge what sort of model might be useful.

Both approaches - a linear (or non-linear) time trend and seasonal dummy variables might be necessary. (Normally dummy variables are used for seasonal or calendar effects or shocks).

If you fit a dummy time variable for every time period and you don't have many observations per time period you could easily end up over fitting. Also, if you use a series of independent dummy variables you have no idea what the effect of the next time period will be - since it will be independent as well. This makes it less useful for forecasting than other ways of using time in a model.

Perhaps an even more complex process such as ARIMA might be useful. Something like the forecast package in R might be useful for understanding the time series. For fitting a model you might want to look beyond OLS and consider auto-regressive or dynamic regression models.

| cite | improve this answer | |
  • $\begingroup$ what if the data is stationary and has no trend and I would like to only control for seasonality? As noted in my problem statement, the error term is formulated as ARMA and this is form of time series dynamic regression. $\endgroup$ – forecaster Feb 4 '14 at 23:07
  • $\begingroup$ For a stationary model an OLS regression with seasonal dummies is easier, and has the advantage of being easy to interpret. But it assumes that the seasonal effect is fixed and uncorrelated with longer time trends. $\endgroup$ – david25272 Feb 4 '14 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.