0
$\begingroup$

I am struggling to understand when a regression variable that is non-lagged would be beneficial to a forecasting algorithm.

I have been investigating the unobserved component model algorithm. I am finding that even when the non-lagged regression variable is linearly correlated with the response, a better model (lower cross-validated MASE) does not use the regression variable. It seems a 'slope' term within UCM could just as easily be used as the regression variable.

For UCM terms: https://www.lexjansen.com/nesug/nesug04/an/an03.pdf

UCM Model

I am also investigating ARIMA models. I am finding again that if the non-lagged regression variable is linearly correlated with the response that this same trend can be estimated just with an ARIMA model. For example, I could use a difference to de-trend the model so it's now stationary. A transfer function utilizing the correlated regression variable is no longer significant.

For ARIMA terms: https://v8doc.sas.com/sashtml/ets/chap7/sect8.htm#:~:text=Thus%2C%20the%20general%20notation%20for,order%20of%20the%20seasonal%20part.

ARIMA Model: enter image description here

I guess the question boils down to, When is the regression variable explaining variation in the response that couldn't easily be estimated by terms expressed in lags of the response?".

Is there a simple guide during EDA of the y & x that I could use to try & identify patterns that a simple lagged y model could not explain?

Thank you in advance for your help, narnia649

$\endgroup$
4
  • 1
    $\begingroup$ Hi: I think it would be better if you wrote down the equations you are referrring to in your explanation. Then myself ( and maybe others ? ) could get a better comprehension of what you're describing. That's not to say you don't explain things clearly but it's the referring to the equations part that I find confusing. Thanks. Also, it's of course possible that someone else may answer without the need for the actual equations. It could be me :). $\endgroup$ – mlofton Mar 15 at 15:48
  • $\begingroup$ Beware of differencing time series that have linear trends but not unit roots. This is a bad idea known as overdifferencing with adverse consequences. $\endgroup$ – Richard Hardy Mar 15 at 16:20
  • $\begingroup$ the only time when non-lagged regressor doesn't help is when it is completely irrelevant, i.e. contains absolutely no useful information. $\endgroup$ – Aksakal Mar 15 at 17:41
  • $\begingroup$ @RichardHardy - Yeah that's a good point. The trend ADF is less than the critical threshold, which from my understanding indicates a first difference would be adequate. $\endgroup$ – narnia649 Mar 15 at 19:23
0
$\begingroup$

Hi: I glanced at the paper and that model has a lot of terms in it so I won't even try to go into all of them. I'll just focus on one issue since it's related to your question. The thing to realize is that, when you use a lagged response variable on the RHS of a model, you really are using past $X_t$, it's just implicit. Take the simplest case, namely, ARDL(1,0). ( this is called an autogressive distributed lag in the econometrics literature ).

$Y_t = \alpha Y_{t-1} + \beta X_{t} + \epsilon_t$.

This can be re-written as:

$Y_{t}(1 - \alpha L) = \beta X_{t} + \epsilon_t$.

Then, dividing both sides by $(1 - \alpha L)$, ( and assuming that $\alpha$ is between 0 and 1 so that the infinite series converges ) results in:

$Y_{t} = \beta \sum_{i=0}^{\infty} \alpha^{i} X_{t-i} + \sum_{i=0}^{\infty} \alpha^{i} \epsilon_{t-i}$.

So, in the simplest case ( I imagine this carries over to the more complicated case where you have multiple lagged terms but i won't try to show it ), it's really just decorative in that it is possible to get rid of the lagged dependent variable. The problem with doing that is that the series is infinite and you'll have difficulties estimating it if the $X_{t-i}$ are highly correlated.

The model I just described is has another name called the "koyck distributed lag" in case you want to find it in the literature. But the point I'm trying to make is that, when you see a lagged dependent variable in a model, it's really just a simpler way of expressing the fact that the response is based on regressor variables and error terms terms that go way back in time.

$\endgroup$
5
  • $\begingroup$ Thanks for your response, that's very helpful! Yes, I did notice that when I have multiple independent variables (X's) I tend to get more convergence issues as well as an error regarding the Hessian Matrix, which I typically see in OLS when there is too much multi-collinearity. If there are two models that have similar Cross Validation MASE (one with x or multiple regressors & one without), would it be better / safer to use the model without the x (don't have to worry about extrapolation, multi-collinearity, etc.) $\endgroup$ – narnia649 Mar 15 at 19:11
  • $\begingroup$ Hi: Don't take anything I say as a hard fast and rule because you're of course doing the analysis. But, when you say, without "x", you're saying that the model without any predictors ( and just lags of the response ) does as well as the one with predictors ? If so, that says that the response is only a function of the past response which I guess is possible ?. Is my understanding correct ? if so, then yes, I wouldn't include regressors but I would make sure that is what's happening. Note that this UCM model that they describe has a lot of things in it so maybe something is going on there. $\endgroup$ – mlofton Mar 15 at 20:16
  • $\begingroup$ Continuing from above: In fact, I wouldn't include all those things that they use such as cycle, trend, etc. Just use the response and the lagged response and any predictors, atleast as a starting point. I have found, atleast in my experience, that parsimony is a good thing and throwing various terms into the model makes things confusing and hard to understand. $\endgroup$ – mlofton Mar 15 at 20:18
  • $\begingroup$ Thanks @mlofton. Yes, without X means no regressor's. In the example I am referring to, I am seeing that a model with a level & slope term can predict the response as well as or better than a model that contains lagged response variables or lagged regressor variables. Yeah, I am using a grid approach to investigate the multiple parameters in a UCM model. Then, I used a five-fold cross-validation to determine which model is performing the best. I am seeing the more variables I add to the model greatly overfits the data and gives high cross-validation error. $\endgroup$ – narnia649 Mar 15 at 23:21
  • $\begingroup$ I can see the level and slope term being enough. Useful explanatory variables are tough to find. Often, all they do is make prediction worse. $\endgroup$ – mlofton Mar 16 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.