1
$\begingroup$

I've got 5+ years of data, with multiple observations per week. I'd like to understand if there is a correlation between my dependent and independent variables.

The catch is that I know this data is highly seasonal, with lows in winter and highs in summer. My concern is that the correlation could be thrown off by this.

My first thought was to group the data by season and perform the correlation within each season, but I assumed there was a better statistical method for this. So far, everything I've seen seems to be based on the idea of rolling the data up by month, running ARIMA or SEATS on the data, and then projecting.

What I haven't seen is how one would apply this to my original problem. Is it legitimate to average my data up by month, use ARIMA or SEATS to get the seasonal component by month, and then subtract that out of each individual element so I can correlate across seasons? If not, any input on how to tackle this problem would be greatly appreciated.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ The basic idea is that you want to understand the relationships over time in the absence of spurious relationships -- to the extent that this is statistically possible. There could be lots of issues lurking in your time series besides seasonality, e.g., autocorrelation, nonstationarity, trends, unit roots, and so on. Luckily, you have lots of data to work with. Look into removing these potential biases by developing "white noise" residuals and modeling that. One basic approach to this is a Holt-Winters decomposition of the data. $\endgroup$
    – user78229
    Commented Jun 30, 2016 at 20:54
  • $\begingroup$ I really appreciate the comment, but the methods in R I've seen that decompose the data as you're describing seem to need the data in aggregate, which makes sense. I'm struggling with how to take that aggregate answer and apply it (if it is statistically valid to do so) back to the raw data. $\endgroup$
    – John Tarr
    Commented Jun 30, 2016 at 21:11
  • $\begingroup$ "Data in aggregate?" What does this mean to you? Also, it would clarify your concerns if you were to add a sample of the data to your question. $\endgroup$
    – user78229
    Commented Jun 30, 2016 at 21:15
  • $\begingroup$ The data is proprietary, and I'm not sure how to add something representative of the amount of data I'm talking about. Aggregation is taking individual elements and combining them, as in, sum, average, etc. My understanding is that you can't just feed these decomposition functions all of the raw data, it must first be aggregated by month, quarter, or year. $\endgroup$
    – John Tarr
    Commented Jul 1, 2016 at 12:03

1 Answer 1

Reset to default
1
$\begingroup$

You say "multiple observations" per week, is this in some irregular pattern? Then you have an unevenly spaced time series, and should look into methods/models for that. Should you aggregate data per week? That cannot really be answered in the abstract. Is it count data? sums of counts is a count, so yes, you should sum (not average). Is it sales data (for some nonoverlapping periods) you should sum.

More general, if this is extensive variables, summing makes sense. Averaging is more problematical, it makes sense conceptually for intensive variables, but if will change the variance/correlation structure of the series, so can lead to misleading results.

If you are able to construct (with advice in first paragraph) a weekly time series, you could look into modeling with arima. But maybe, in either case, start with some time series decomposition method. See also the comment by @DJohnson:

The basic idea is that you want to understand the relationships over time in the absence of spurious relationships -- to the extent that this is statistically possible. There could be lots of issues lurking in your time series besides seasonality, e.g., autocorrelation, nonstationarity, trends, unit roots, and so on. Luckily, you have lots of data to work with. Look into removing these potential biases by developing "white noise" residuals and modeling that. One basic approach to this is a Holt-Winters decomposition of the data.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.