0
$\begingroup$

Im trying to estimate the best model for my dataset using R. I tried both auto.arima function as well as AIC to identify the best model (I created several models and picked the best one according to Akaike). Both "best" models were, of course, the same, which happened to be an ARIMA(1,1,1). The issue is that one of the coefficients (the MA) is not significant (absolute value of the estimated value divided by its sd is less than 2.093). What should I do in this case?

Thanks in advance

EDIT: If I use BIC instead of AIC, the 'best' model is an ARIMA (1,1,0), which is consistent with the fact that the MA term in the 1,1,1, is insignificant.

$\endgroup$
1
  • $\begingroup$ AIC is notoriously "liberal". It keeps non-significant terms all the time. Even though the "AIC approved" model is harder to interpret, it may lead to somewhat better out-of-sample performance than the "best" model where all the terms are statistically significant at the 5% significance level. Depends on the context of the problem... What you should do is choose one model selection criterion and be loyal to it until the end of the model selection process. I personally would not have chosen AIC in the first place. $\endgroup$
    – stans
    Commented Jul 1, 2018 at 22:49

1 Answer 1

-1
$\begingroup$

Bravo @stans , auto.arima in a brute force list-based procedure that tries a fixed set of models and selects the calculated AIC based upon estimated parameters. The AIC should be calculated from residuals using models that control for intervention administration, otherwise the intervention effects are taken to be Gaussian noise, underestimating the actual model's autoregressive effect and thus miscalculates the model parameters which leads directly to an incorrect error sum of squares and ultimately an incorrect AIC. Most SE responders do not point out this assumption when they promote the free auto.arima tool which I think is a serious error of omission.

Modern/Correct/Advanced ARIMA time series analysis is conducted by identifying a starting model and then iterating to refine the initially suggested model as detailed in If I am convinced that a series is mostly trend+season, what is it I should check about the residuals? will be of help here.

If you wish please post your data I will illustrate this for you and the possible failings/omissions of auto.arima.

As an example of a very bad model identification using auto.arima see https://www.omicsonline.org/open-access/an-implementation-of-the-mycielski-algorithm-as-a-predictor-in-r-2090-4541-1000195.php?aid=65324 .

EDITED AFTER RECEIPT OF DATA:

I took your 200 observations and introduced them to AUTOBOX. Here is the original data enter image description here . AUTOBOX automatically detected a change in model parameters at period 62 thus suggesting 2 regimes 1-61 and 62-200 which is (NOW !) visually obvious from the graph. This was based on the CHOW test for constancy of parameters.

The model is here enter image description here with stats here enter image description here . Note that there is 1 auto-regressive coefficient at lag 2 .

The Actual/Fit and Forecast is here enter image description here with residual plot here enter image description here and acf here enter image description here suggesting model sufficiency.

The Fit and Forecast graph nicely presents the impact of the equation .enter image description here

In summary the reason why your simple attempts to form a model with significant structure failed is that either the parameters changed at period 61 OR the error variance changed at period 62 creating OBFUSCATION in the acf yielding twice as much OBFUSCATION in attempting to torture/use the sample acf with a restricted set of models/solutions .

Is this simulated data ?

$\endgroup$
7
  • 1
    $\begingroup$ The core of the question was "why does a model selected using AIC have coefficients that are 'not significant'"? I don't think you addressed that. $\endgroup$ Commented Jul 2, 2018 at 2:26
  • $\begingroup$ Thanks @Stans and IrishStat for your replies. I upload in this link -> mediafire.com/file/4bqz2gq8mkwzora/SimulatedTS28.csv/file the dataset im working with. It's a small, simulated one. $\endgroup$
    – BorisD
    Commented Jul 2, 2018 at 3:11
  • $\begingroup$ @TheLaconic Yep. The question is whether that is a problem or not. $\endgroup$
    – BorisD
    Commented Jul 2, 2018 at 3:54
  • $\begingroup$ @BorisD I am unable to use the download site ...so please email the series to me at [email protected] $\endgroup$
    – IrishStat
    Commented Jul 2, 2018 at 12:02
  • $\begingroup$ @IrishStat Done, sir. $\endgroup$
    – BorisD
    Commented Jul 2, 2018 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.