3
$\begingroup$

I am using a publicly available data Kaggle: Rossmann Store Sales and trying to forecast sales. I am using Python.

My timeseries is stationary, confirmed via the Dickey-Fuller test. However, I wanted to perform seasonal decomposition.

I performed seasonal decompositions using statsmodels.tsa.seasonal.seasonal_decompose. And my seasonal decomposition looks like this: enter image description here

When I plot ACF of residuals there appears to be too much autocorelation! enter image description here

Am I doing something wrong? or looking at it the wrong way? My understanding is residuals should show no autocorelation because trend and seasonal have been taken out or adjusted for.

Update 1: Using freq=13 I perform seasonal decomposition and ACF of residuals is given below: enter image description here

Update 2: As requested by @IrishStat, I am posting the original data

Head(10):
Date
2013-01-01       0
2013-01-02    5737
2013-01-03    5292
2013-01-04    5623
2013-01-05    5018
2013-01-06       0
2013-01-07    9277
2013-01-08    7479
2013-01-09    6681
2013-01-10    6680
Name: Sales, dtype: int64

This is the plot of original data:

enter image description here

Improve this question
$\endgroup$
7
  • $\begingroup$ It looks like you've taken a period of ~26 but it seems to be the half : ~13. Try with that and show your results. $\endgroup$
    – el Josso
    Jul 11, 2016 at 7:22
  • $\begingroup$ @el Josso, are you refering to parameter freq in seasonal_decompose? The figures were initially generated by using freq=30, I have updated with freq=13. $\endgroup$
    – user2979010
    Jul 11, 2016 at 11:34
  • $\begingroup$ Try 15 or something like that. On your first figure, You can see that you are taking one period for two. $\endgroup$
    – el Josso
    Jul 11, 2016 at 12:10
  • $\begingroup$ Sales are always seasonal and autocorrelated. If your model structure does not account for autocorrelation, then it'll show up in residuals - where else could it go?! Also sales are usually non-stationary if you measure them in currency. They're usually stationary only in short term. $\endgroup$
    – Aksakal
    Jul 11, 2016 at 13:54
  • $\begingroup$ @el Josso, can you please elaborate what do you mean by "one period for two"? $\endgroup$
    – user2979010
    Jul 11, 2016 at 23:25

2 Answers 2

Reset to default
7
$\begingroup$

You disaggregate a time series into three components -- trend, seasonal and residual.

  • The trend component is supposed to capture the slowly-moving overall level of the series.
  • The seasonal component captures patterns that repeat every season.
  • The residual is what is left. It may or may not be autocorrelated. For example, there can be some autocorrelated pattern evolving quickly around the slowly moving trend plus the seasonal fluctuations. This kind of pattern cannot be ascribed to the trend component (the former moves too fast) or the seasonal component (the former does not obey seasonal timing). So it is left in the remainder.

See also section "Time series decomposition" from Hyndman & Athanasopoulos "Forecasting: principles and practice".

Improve this answer
$\endgroup$
6
  • $\begingroup$ Considering the residuals to be stationary (it appears to be) I can then use ARMA model to forecast the residuals. How do I go about determining the MA term for my ARMA in this instance? My understanding is that MA term can be determined by ACF plot. $\endgroup$
    – user2979010
    Jul 11, 2016 at 11:42
  • $\begingroup$ Yes, you could try forecasting with an ARMA model. Material on determining ARMA orders can be found in (almost all) time series textbooks and has also been discussed extensively in other threads at Cross Validated; I suggest you look them up. ACF and PACF plots can be helpful in simple cases (pure AR or pure MA processes) but become less useful in more complicated cases (both AR and MA parts present). $\endgroup$
    – Richard Hardy
    Jul 11, 2016 at 11:51
  • $\begingroup$ do you consider the residuals in the problem a complicated case? Would it be possible to throw some light on what techniques might be better fit for such a case? $\endgroup$
    – user2979010
    Jul 11, 2016 at 13:39
  • $\begingroup$ @user2979010, This is a new question, please post it as such (i.e. given certain data -- in your case the remainder term from decomposition -- how to model them best with the goal being forecasting). The original question asks why the remainder term after time series decomposition is autocorrelated. That question I have answered (probably not to your complete satisfaction, that I cannot know). $\endgroup$
    – Richard Hardy
    Jul 11, 2016 at 13:43
  • $\begingroup$ sure, I will post a new question. $\endgroup$
    – user2979010
    Jul 11, 2016 at 23:26
2
$\begingroup$

Correct decomposition requires a good model which might have either fixed seasonal effects or seasonal auto-regressive effects and one or more trends ( or level shifts ) and constant error variance and constant parameters over time. Perhaps your data set ( which you should post) doesn't conform to the requiremnents/assumptions and thus needs "special handling" or empirically-based model formulation.

Improve this answer
$\endgroup$
8
  • $\begingroup$ I have added the original data plot $\endgroup$
    – user2979010
    Jul 11, 2016 at 13:37
  • $\begingroup$ Please post the entire data set . $\endgroup$
    – IrishStat
    Jul 11, 2016 at 19:18
  • $\begingroup$ The dataset is publicly available via Kaggle Link: kaggle.com/c/rossmann-store-sales $\endgroup$
    – user2979010
    Jul 11, 2016 at 23:23
  • 1
    $\begingroup$ click on the link, go to "Dashboard --> Data" on the left. The "Data" page will open with links to data CSV files. You will need to login to Kaggle to access the data. I don't think Kaggle T&C allows to re-host/re-distribute the data therefore I cannot post it here. $\endgroup$
    – user2979010
    Jul 12, 2016 at 3:27
  • 1
    $\begingroup$ You need to post them as I need to have the values before I constructively critique what you are doing ( or not doing ! ). There is no reason that you should not . $\endgroup$
    – IrishStat
    Jul 14, 2016 at 10:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.