2
$\begingroup$

I have a set of time series from different runs of a computational model. I know that the computational model is highly historical in the sense that the state early in a trial heavily influences (in a non-linear, difficult to summarise way) what the state will be later in that same trial. Accordingly, I understand the data in these time series to be non-independent.

My goal is to show that when I change a particular parameter in the model from Value A to Value B, it has a consistent influence upon the dynamics. When I plot histograms of the state variables, it is clear to me that this is the case, but I am looking for a statistical test that will allow me to say so with less subjectivity.

I don't know how to do so. I have searched around for non-independent statistical tests, but most of what I find are tests for detecting if your data is independent or not. I know that my data is not independent, and this non-independence is an essential feature of my model, so I can't get rid of it.

I have 10 trials of each parameter value. I thought I might divide each of those sets into two groups of 5 trials, thus:

  • Data Set # 1 : Parameter Value A, trials 0-4
  • Data Set # 2 : Parameter Value A, trials 5-9
  • Data Set # 3 : Parameter Value B, trials 0-4
  • Data Set # 4 : Parameter Value B, trials 5-9

I then could (somehow!) show that Data Set #1 is more similar to Data Set #2 than the amalgamation of Data Set #1 and #2 is to the amalgamation of Data Sets #3 and #4.

...and similarly show that Data Set #3 is more similar to Data Set #4 than the amalgamation of Data Set #1 and #2 is to the amalgamation of Data Sets #3 and #4.

Is that a good plan? If so, what measures of similarity should I learn about that can be applied to non-independent data?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

You say: "My goal is to show that when I change a particular parameter in the model from Value A to Value B, it has a consistent influence upon the dynamics." For doing that, you need some metric $M$ (e.g. some numeric value) that describes this "influence upon the dynamics". E.g., if the change in dynamics is a change in variance (fluctuation, volatility) in the time series, then this would be your metric $M$.

Once you have that, the rest is fairly standard statistics. You simply check whether the ten values of $M$ obtained with A are significantly different from the ten values of $M$ obtained with B. Depending on what you can assume about the data, you could look into e.g. the t-test or the Mann–Whitney U test.

Note, that the question of what inner dependencies (auto-correlation) your time series are having is not really relevant in this approach.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for this answer. I'll accept it as the answer, but also, a quick follow up: what if I don't know what the effect of the parameter is. Is there a way to check that there is "a difference" without pre-specifying that it is the variance (or volatility or whatever)? TIA $\endgroup$
    – weemattisnot
    Commented Aug 26, 2022 at 3:38
  • 1
    $\begingroup$ You always need some dependent variable. If you cannot really describe what you mean by "dynamics", check out some of the standard properties of time series, like trend, auto-correlation, volatility, heteroskedasticity, seasonality, ... Also, you could fit a time series model to both cases and compare the fitted model parameters. $\endgroup$
    – frank
    Commented Aug 26, 2022 at 4:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.