I am currently generating data by simulating a model of chemical system under different conditions (temperature) over time. In each simulation, the starting structure being modeled is exactly the same - only the temperature is different. The system is allowed to propagate over time and the length and number of observations in each simulation is identical. I would like to compare mean values e.g. distances between two atom under the different conditions. I have two questions:

  1. Should I regard the two simulations I have (high and low temperature) as paired data. How would an analogous human study be treated (e.g. comparing the behaviour of a single human participant under during a 1 hour period under condition 1 and another 1 hour period after an extensive washout period - under condition 2)?

  2. Since I effectively have two time series, what are the implications of the distance I want to measure being autocorrelated in some way?


Stochastic model

Consider the following scenario:

  • You measure the blood pressure of the same man every day for two weeks;
  • You measure the blood pressure of the same woman every day for two weeks;

What can you say about gender and blood pressure. Answer: not very much. This example is very similar to what you have. Until your sample size is bigger than one, you can't say very much.

As I commented above, you know that your two simulations are different. If you simulated the process enough times, you would get a "small p-value". What I suspect you want is to estimate the magnitude of these differences. So,

  • You need more simulations to estimate the variance. There's no way around this.
  • At each time-point store $x$ and $x^2$. This will allow you estimate the mean and variance of $x$.
  • Once you have the mean and variance you could plot:
    • $x$ with a 95% confidence interval over time;
    • $x_H - x_L$ with a 95% confidence interval over time.

Deterministic model

Consider the following deterministic model for a simple death process:

\begin{equation} \frac{dX(t)}{dt} = -\mu X(t) \end{equation}

We can solve this equation to get $X(0) e^{-\mu t}$. Now suppose:

  • For a high temperature, X(0) = 50
  • For a low temperature, X(0) = 30.

Then for all $t$, the population of the high temperature is always greater than the low temperature. As we have a deterministic model, we have no uncertainty. If you did a t-test, your variance would be zero.

This is your scenario. Your model is deterministic. So the simulations are either the same or they are different. You have no uncertainty in the result.

  • $\begingroup$ OK - sorry its a bit unclear. I have a simulation based on physical laws which models how two molecules move about in bath of water. One simulation (1ns) long models what would happen if the system is at 0 degrees and a second simulation models what would happen at 25 degrees C. The output is effectively an animation that I can both watch and take detailed measurements on e.g. I can measure distances between atoms to see how closely the molecules approach each other . I want to know the effect of temperature on how close the two molecules approach each other. $\endgroup$
    – N26
    Jul 4 '11 at 19:10
  • $\begingroup$ To rephrase my question - is it safe to do a ttest to compare the mean distance in the two simulations? If not, why not? If so, should the data be treated as paired? I have limited statistical knowledge of time series - and am really asking - does the fact that I am dealing with time series imply that "a special kind of statistical approach" should be taken? $\endgroup$
    – N26
    Jul 4 '11 at 19:15
  • $\begingroup$ But why do you want to do a t-test. Just simulate the process N times and calculate the difference. A better method would be to simulate the difference over time, and plot the difference with associated error $\endgroup$ Jul 5 '11 at 16:44
  • $\begingroup$ The output of my simulation is effectively two time series. Time series 1 is the distance between atom A and atom B over a 1ns time period at low temperature. Time series 2 is the distance between atom A and atom B over a 1ns time period at high temperature. I want to know if temperature affects the average distance between the two atoms, so it seemed sensible to compare the mean distance over time series 1 with the mean distance over time series 2 using a t-test. As this is resource intensive (i.e. takes weeks to generate 1 time series) I can only simulate each temperate once. $\endgroup$
    – N26
    Jul 6 '11 at 13:32
  • $\begingroup$ I just wanted to say thanks for all your suggestions and input. My level of clarity hasn't been great ^^. I guess if I were to give a real life analogy it would be: measure Bob's blood pressure over two weeks when he is on holiday in Spain, then measure Bob's blood pressure during final two weeks of being a PhD student. What is the effect of stress on Bob's blood pressure? Bob happens to be some really important dude and for some reason, we care about the idiosyncrasies of this one person. The simulation is deterministic. $\endgroup$
    – N26
    Jul 7 '11 at 20:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.