3
$\begingroup$

I have two time series data sets which contain hourly-intervalled, monthly, and yearly household electricity consumption in kWh. One data set is produced by a simulation, the other gathered from the real-world. My aim is to validate the simulated output by using the data gathered from the real-world.

I want to measure the similarity between these data sets, and be able to say if these are statistically similar. My first intuition is to use a correlation coefficient such a Pearson product moment correlation. But from what I read in previous posts that in general the correlation coefficient between two time-series may be a very poor metric.

I'm not very keen on statistics related to time series, but would something like a cross-correlation or maybe ARIMA do the trick? Could someone please point me in the direction for a technique which I can use in SPSS?

Improve this question
$\endgroup$
6
  • $\begingroup$ Quantifying similarity is always possible. However, correlation measures linearity of relationship, not similarity of values (the correlation between $y$ and $by$ is identically 1 if $b$ is positive, but for large $b$ the two don't qualify as "similar". Concordance correlation does measure agreement. For most purposes, however, looking at the pattern of differences between observed and simulated will be as or more informative than trying to encapsulate similarity in a single number. Cross-correlation or ARIMA sound like overkill for 24 values. $\endgroup$
    – Nick Cox
    Oct 8, 2013 at 13:57
  • $\begingroup$ Thank for your informative answer Nick Cox. Concordance correlation does seem like a good alternative. I understand from your explanation that measuring correlation might be useless since it measures linearity of relationship, and doesn't take into account the impact of larger values. I have no experience with Cross-correlation or ARIMA, but could you maybe explain to me why Cross-correlation and ARIMA are overkill to measure similarity? Aren't these tools more suitable to measure similarity and at the same time provide more details? $\endgroup$
    – Rez
    Oct 8, 2013 at 14:28
  • $\begingroup$ Cross-correlation may have some descriptive merit if it exposes a lag structure e.g. if your simulated values lead or lag the observed values, but that should be evident too from plotting the data. On ARIMA: I am not sure that you understand what it does. It's a class of models; so do you intend to model the structure of disagreement between your simulation model and the observed values? For 24 hourly values, I wouldn't start with ARIMA, and that's a small dataset any way. $\endgroup$
    – Nick Cox
    Oct 8, 2013 at 14:36
  • $\begingroup$ ARIMA is mainly used for modeling? I was advised that I could compare ARIMA coefficients. But I guess that I was pointed into the wrong direction. Do you by any chance know of any chance happen to know any useful descriptive statistics to examine such as standard deviation, median, and so on. Thank you for your help. $\endgroup$
    – Rez
    Oct 8, 2013 at 15:08
  • $\begingroup$ You seem fixated on producing descriptive measures. I suggest that you focus much more on looking at your data. As you explained your problem, you have two time series, each 24 observations long. Plot the two series against time, and the difference between observed and simulated against time. You could e.g. compare the means (do they agree? how much do they differ?) and/or the differences between the means as well, but I'd advise against the idea that any single named measure gives you the answer. Note that getting the same means even would not establish that the series were identical. $\endgroup$
    – Nick Cox
    Oct 8, 2013 at 15:14

2 Answers 2

Reset to default
1
$\begingroup$

Old thread but I'm adding this for others who may come across it. Cross correl is a fantastic tool for observing whether peaks in one series kind of 'lead' peaks in another series, so if you think there may be a predictor and response, but syncopated. For 24 observations it's fine, but your critical values (to be judged significant) are just higher, like with p-values for t-tests. Very useful but since the two datasets can't be a predictor and different variable response, maybe not in this case.

For short time-series, arima needs 16 observations. It does show seasonality, dependance on adjacent observations, moving averages, etc. brilliant tool and spss has good tutorials. Minitab and SPSS pretty easy to use. Don't be put off by expertise, just give these things a try if you have time to read up, and they help you understand you data even if you don't use the eventual models.

Improve this answer
$\endgroup$
-4
$\begingroup$

I have never used SPSS so I don't know if the Kolmogorov Smirnov test is implemented there. If it is, check it out. It will solve your problem

Improve this answer
$\endgroup$
5
  • $\begingroup$ This is disputable. Kolmogorov-Smirnov at most tests for similarity of distributions, not similarity observation by observation. The dependence structure expectable in time series is also problematic. $\endgroup$
    – Nick Cox
    Oct 8, 2013 at 13:32
  • $\begingroup$ Thank for your reply Alexandra Paun. But I'm probably looking another similarity measure. One that takes into account the autocorrelation, seasonality and such. I think the type which Nick Cox pointed out. $\endgroup$
    – Rez
    Oct 8, 2013 at 13:57
  • $\begingroup$ How is seasonality relevant to 24 hourly values? $\endgroup$
    – Nick Cox
    Oct 8, 2013 at 14:36
  • $\begingroup$ Sorry for the confusion. The simulated data on household energy consumption has been produced for different time frames (years, months) for different time-intervals. I will probably also examine monthly intervals (12 observations). However, I'm not entirely sure if the seasonality effects are present in these scenarios. $\endgroup$
    – Rez
    Oct 8, 2013 at 15:18
  • $\begingroup$ Not explaining your real problem directly just confuses everyone. You said "hourly-intervalled" and "24 observations". Please edit your question before anyone else is misled. $\endgroup$
    – Nick Cox
    Oct 8, 2013 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.