enter image description here

enter image description here

hey, I have a dataset that contains hourly wind speeds. When I plotted my original acf, it showed seasonality at every 24 lags so I applied a difference of 24 to remove seasonality and another difference to remove trends. The diagrams shows the resulting acf and pacf, how do I determine the order of the arima model. Also I am seeing that at every 24 lags in the pacf there is a spike and decaying slowly, is this suppose to happen or does my dataset still contains stationarity?

Original Dataset

  • $\begingroup$ show the original ACF/PACF $\endgroup$ – Aksakal Mar 19 '18 at 17:48
  • $\begingroup$ That does look very puzzling. What meteorology is it that implies that raw autocorrelations are negative at lag 1 hour and strongly negative at lag 24 hours (but negligible at 48(24)whatever hours)? Can you show the original data (or a sample of them)? $\endgroup$ – Nick Cox Mar 19 '18 at 17:50
  • $\begingroup$ There is simple jargon for this that isn't even jargon: variation with time of day. Why call it seasonality? $\endgroup$ – Nick Cox Mar 19 '18 at 18:03
  • $\begingroup$ Why don't you post the actual data ? $\endgroup$ – IrishStat Mar 20 '18 at 9:45

A good starting point can be the forecast package of prof Hyndman in R. The auto.arima() function can give you a suitable model and this can be a good starting point of your investigation. I have learned a lot from his open book https://www.otexts.org/fpp/

  • $\begingroup$ Does it handle seasonality? $\endgroup$ – Michael R. Chernick Mar 19 '18 at 6:12
  • 1
    $\begingroup$ yes it does. various aspects like max seasonal differencing etc. can be explicitly given as argument. documentation $\endgroup$ – Pankaj Joshi Mar 19 '18 at 6:44
  • $\begingroup$ unfortunately it assumes no pulses , no level shifts , no seasonal pulses , no local time trends, constant parameters over time , consyabt error variance over time .... $\endgroup$ – IrishStat Mar 19 '18 at 12:44
  • $\begingroup$ Unfortunately I am also pretty new to this forecasting business, I have learned a lot from your comments on this site. If you could elaborate how can the aforementioned be incorporated/handled it would be very helpful. $\endgroup$ – Pankaj Joshi Mar 19 '18 at 13:13
  • $\begingroup$ stats.stackexchange.com/questions/317734/… where @adamo succinctly and quite correctly reflected "The correlogram should be calculated from residuals using a model that controls for intervention administration, otherwise the intervention effects are taken to be Gaussian noise, underestimating the actual autoregressive effect." $\endgroup$ – IrishStat Mar 19 '18 at 14:37

In response to Pankaj Joshi (thanks for asking !): See Interrupted Time Series Analysis - ARIMAX for High Frequency Biological Data? where @AdamO correctly reflected/opined "The correlogram should be calculated from residuals using a model that controls for intervention administration, otherwise the intervention effects are taken to be Gaussian noise, underestimating the actual autoregressive effect.". This comment reflects the fact that if there are interventions present they effectively mask the true (but unknown) arima structure. Prof. J.K.Ord once commented to me that is is like the "Alice in Wonderland effect".

Interventions must be specified either by the user or detected by Intervention Detection procedures. The problem/opportunity is to simultaneously identify BOTH arima structure & the Interventions as is done by AUTOBOX, a piece of software that I have helped to develop.

Since auto.arima naively assumes no Intervention series and constant parameters over time and constant error variance over time it is easily confused and delivers inadequate results which is frequently cited in these pages .

Complete transparency suggests that I mention https://cran.r-project.org/web/packages/tsoutliers/index.html as a tool to identify interventions which requires the user to pre-specify the form of the ARIMA model and fails to consider time varying parameters and time varying error variance.

So the erroneously suggested "logic" goes .. use a two step approach STEP 1:... use auto.arima to identify the ARIMA portion assuming no interventions present in the data THEN in STEP 2 identify the interventions that were assumed not to be present in the first step. Patch the two answers together !

If you wish you can post your data and I will report the results of a simultaneous identification strategy where embedded interdependent heuristics are employed to simultaneously suggest a possibly useful answer.

  • $\begingroup$ Is there a way I can upload my dataset, it contains 14376 points? When I use the auto.arima in R it gives me a (0,1,1) model, however when I plot the acf of the residuals you can clearly see a periodic output. Any help? $\endgroup$ – L N Mar 21 '18 at 21:46
  • $\begingroup$ hey i sent the document with the hourly wind speed data for you. $\endgroup$ – L N Mar 21 '18 at 22:44
  • $\begingroup$ . I took the first 2400 and obtained a model that had an AR(1) and a few hourly dummies … There doesn’t appear to be any justification for any differencing whatsoever for the first 2400 values. Visually the first 2400 look like rest of jhe series thus I would go with seasonal dummies and an ar(1) plus pulse indicators $\endgroup$ – IrishStat Mar 21 '18 at 23:29
  • $\begingroup$ Finally .. if a series has a level shift (determinstic structure is present) the ACF will suggest non-stationarity (which is true). The correct remedy is to de-mean the series NOT to difference the series. The moral of my story is that a series may be non-stationary as suggested by the correlogram BUT the remedy is not necessarily differencing. Your series is driven by strong seasonal dummies (deterministic structure) and should be modelled using them $\endgroup$ – IrishStat Mar 22 '18 at 1:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.