Is it valid to aggregate a time series to make it look more meaningful?

Question

Another question about time series from me.

I have a dataset which gives daily records of violent incidents in a psychiatric hospital over three years. With the help from my previous question I have been fiddling with it and am a bit happier about it now.

The thing I have now is that the daily series is very noisy. It fluctuates wildly, up and down, from 0 at times up to 20. Using loess plots and the forecast package (which I can highly recommend for novices like me) I just get a totally flat line, with massive confidence intervals from the forecast.

However, aggregating weekly or monthly the data make a lot more sense. They sweep down from the start of the series, and then increase again in the middle. Loess plotting and the forecast package both produce something that looks a lot more meaningful.

It does feel a bit like cheating though. Am I just preferring the aggregated versions because they look nice with no real validity to it?

Or would it be better to compute a moving average and use that as the basis? I'm afraid I don't understand the theory behind all this well enough to be confident about what is acceptable

Answer 1

This totally depends on your time series and what effect you want to discover/proof etc.

An important thing here is, what kind of periods do you have in your data. Make a spectrum of you data and see what frequencies are common in you data.

Anyway, you are not lying when you decide to display aggregated values. When you are looking to effects that are occurring over weeks (like, more violence in summer when it's hot weather) it is the right thing to do.

Maybe you can also take a look at the Hilbert Huang Transform. This will give you Intrinsic Mode Functions that are very handy for visual analyses.

Answer 2

It's very common in forecasting to aggregate data in order to increase the signal/noise ratio. There are several papers on the effect of temporal aggregation on forecast accuracy in economics, for example. What you're probably seeing in the daily data is a weak signal that is being swamped by noise, whereas the weekly and monthly data are showing a stronger signal that is more visible.

Whether you want to use temporal aggregation depends entirely on what your purpose is. If you need forecasts of daily incidents, then aggregation isn't going to be much use. If you are interested in exploring the effects of several covariates on the frequency of incidence, and all your data are available on a daily basis, then I would probably use the daily data as it will give a larger sample size and probably enable you to detect the effects more easily.

Since you are using the forecast package, presumably you are interested in time series forecasting. So do you need daily forecasts, weekly forecasts or monthly forecasts? The answer will determine whether aggregation is appropriate for you.

Answer 3

The problem (dilemma) you face appears to be the one of selecting an optimal (or otherwise good) sampling interval for revising your forecasts. To start with, see link text of Brown's famous book, which would also qualify as a good reference. It all boils down to "balancing the risk of not noticing a change quickly against the inherent variability of the data and the cost of revising plans frequently". If you are not prepared to revise your forecast (and the decisions that motivated it) daily, you don't really need to use the (noisiest) daily data. An important point, often lost in the contemporary forecasting literature, is that forecasts are only necessary to assist with making a decision (unless one also knows how to derive fun from them).