Compare model fitting results between datasets

Question

How should I perform a statistical comparison between the model fitting results on two separate datasets? The same model is applied to both sets, but the data are from two conditions, e.g. control and treatment.

I read that I need an F-test. But the formula for the F statistic (from wikipedia),

$$ F = \frac{\frac{RSS_1 - RSS_2}{p_2-p_1}}{\frac{RSS_2}{n-p_2-1}} $$ with $p$ as the number of parameters in a model and $RSS$ as the sum of residuals.

But in this formula, the models are the same, so $p_2 - p_1$ = 0, and then the formula isn't valid. So where am I going wrong?

Context:

• My model equation is y = s-(s/((a+x)*b)),
• I'm using the fitting procedure from lmfit.Model to find the parameters. (why lmfit? The rest of our processing is already in python, and I couldn't get the statsmodels package to handle general model specifications with parameters; but identifying the statistical method is the primary reason for asking)

EDIT: additional context as requested in comment

• Overall aim: the dynamics of the network are for each node in the network to settle into a decision. The individuals can choose from one of two states, and we are interested in the conditions under which:
  • a) the fraction of nodes in network reaching the same state is maximised, and
  • b) how fast this happens.

Form of data: If we label the control experimental group #1 and the treatment group #2, the data looks like:

# time, average fraction of nodes with majority choice, experimental group

60,  0.52, 2
70,  0.53, 2
80,  0.55, 2
90,  0.54, 2
...

1200, 0.92, 2

• The "average fractions to majority choice" are averages across the replicates carried out.

• We have 15 replicates or more per treatment, and the analysis is on data already obtained.

• group #1 has lower average fractions by the end of the experiment (because they don't have the same level of agreement in general).

• The curve following a saturation equation (above) fits the data reasonably well and fits our hypotheses for the underlying dynamics. So we are not looking to select between models.

• However, we do want to say whether the `treatment' makes a difference.

So, the question here is whether difference between the fitted curves is statistically significant. What is the most appropriate way to test this?

Answer 1

It looks like what you really want to do is just to check (a) whether the mean time significantly differs between conditions and (b) whether the mean "average fraction of nodes with majority choice" significantly differs between conditions. Your model isn't necessary for either of these. For (a), you could use the Mann-Whitney $U$-test, or an independent-samples $t$-test on the logged time. For (b), you could use the Mann-Whitney again or Fisher's exact test (for the latter, you'll need to recode the fractions into node counts).

I still can't tell what the study is about, what the network means or comes from, how the stated model equation relates to the data, what the experimental manipulation is, etc., so it's possible I'm misunderstanding your situation.