# Statistical test to check whether several datasets may be approximated with a single fit

I have a system that simulates some physical phenomenon. The true behavior is assumed to be described with a linear function $$x(t) = a t + b$$. Due to the randomized nature of the simulation each run produces different values for $$x$$.

Now I have several runs sampled at evenly spaced time points $$t_i$$. I would like to check a hypothesis that all the runs simulate the same behavior up to some random fluctuations, that is $$y^k_i = a t_i + b + \varepsilon_i^k$$, where $$k$$ is the number of the run and $$\varepsilon$$ — some random error.

From now I see two possibilities of such testing:

1. Fit every sequence $$(t_i, y^k_i)$$ with a linear trend obtaining individual $$a_k, b_k$$ and somehow compare $$a_k, b_k$$ with fit's RSS. This sounds promising.
2. Fit all the data with single fit, subtract it from all the data and perform testing on the residuals. The idea is that this single fit should be close to the true behavior (under the null hypothesis) and the residuals may be compared as independent values. This obviously won't work if the runs are very different.

My question: is this problem (or similar) known and studied in the literature? I appreciate any references.

UPD. The original question was unclear, so I'd add some real life example. Consider we have two groups of people (males and females). For each group we have meashured the height $$h$$ and the shoe size $$s$$ of each member. From the observations we deduce that shoe size changes linearly with height, i.e. $$s \approx a h + b$$. How can we now check whether this linear law is same or different for men and women?

• Is there some reason why you can’t answer this question based on the parameter settings for each of the simulations?
– EdM
Sep 7, 2019 at 20:39
• The parameters of the simulation itself are fixed. The idea is to check whether simulation results are reproducible or just random Sep 8, 2019 at 8:07
• This seems like you are just testing whether your randomization worked. Sep 8, 2019 at 12:38

It seems like a perfect solution for your problem can be found via the lack-of-fit test. The test requires that more than one observation is available for some time points $$t_i$$, which seems to be the case for your problem since (I believe) each run $$k$$ has an observation at time point $$t_i$$. The null hypothesis is that $$H_0: \exists \, a, b \in \mathbb{R}\,\mathrm{ such \,that } \, \mathbb{E}[y_i^k] = a t_i + b \, \forall i, k,$$ and iid normality of the errors is assumed.