# How can one evaluate the similarity of non-independent data-sets

I have a set of time series from different runs of a computational model. I know that the computational model is highly historical in the sense that the state early in a trial heavily influences (in a non-linear, difficult to summarise way) what the state will be later in that same trial. Accordingly, I understand the data in these time series to be non-independent.

My goal is to show that when I change a particular parameter in the model from Value A to Value B, it has a consistent influence upon the dynamics. When I plot histograms of the state variables, it is clear to me that this is the case, but I am looking for a statistical test that will allow me to say so with less subjectivity.

I don't know how to do so. I have searched around for non-independent statistical tests, but most of what I find are tests for detecting if your data is independent or not. I know that my data is not independent, and this non-independence is an essential feature of my model, so I can't get rid of it.

I have 10 trials of each parameter value. I thought I might divide each of those sets into two groups of 5 trials, thus:

• Data Set # 1 : Parameter Value A, trials 0-4
• Data Set # 2 : Parameter Value A, trials 5-9
• Data Set # 3 : Parameter Value B, trials 0-4
• Data Set # 4 : Parameter Value B, trials 5-9

I then could (somehow!) show that Data Set #1 is more similar to Data Set #2 than the amalgamation of Data Set #1 and #2 is to the amalgamation of Data Sets #3 and #4.

...and similarly show that Data Set #3 is more similar to Data Set #4 than the amalgamation of Data Set #1 and #2 is to the amalgamation of Data Sets #3 and #4.

Is that a good plan? If so, what measures of similarity should I learn about that can be applied to non-independent data?

You say: "My goal is to show that when I change a particular parameter in the model from Value A to Value B, it has a consistent influence upon the dynamics." For doing that, you need some metric $$M$$ (e.g. some numeric value) that describes this "influence upon the dynamics". E.g., if the change in dynamics is a change in variance (fluctuation, volatility) in the time series, then this would be your metric $$M$$.

Once you have that, the rest is fairly standard statistics. You simply check whether the ten values of $$M$$ obtained with A are significantly different from the ten values of $$M$$ obtained with B. Depending on what you can assume about the data, you could look into e.g. the t-test or the Mann–Whitney U test.

Note, that the question of what inner dependencies (auto-correlation) your time series are having is not really relevant in this approach.

• Thanks for this answer. I'll accept it as the answer, but also, a quick follow up: what if I don't know what the effect of the parameter is. Is there a way to check that there is "a difference" without pre-specifying that it is the variance (or volatility or whatever)? TIA Commented Aug 26, 2022 at 3:38
• You always need some dependent variable. If you cannot really describe what you mean by "dynamics", check out some of the standard properties of time series, like trend, auto-correlation, volatility, heteroskedasticity, seasonality, ... Also, you could fit a time series model to both cases and compare the fitted model parameters. Commented Aug 26, 2022 at 4:52