How should I perform a statistical comparison between the model fitting results on two separate datasets? The same model is applied to both sets, but the data are from two conditions, e.g. control and treatment.
I read that I need an F-test. But the formula for the F statistic (from wikipedia),
$$ F = \frac{\frac{RSS_1 - RSS_2}{p_2-p_1}}{\frac{RSS_2}{n-p_2-1}} $$ with $p$ as the number of parameters in a model and $RSS$ as the sum of residuals.
But in this formula, the models are the same, so $p_2 - p_1$ = 0, and then the formula isn't valid. So where am I going wrong?
Context:
- My model equation is
y = s-(s/((a+x)*b))
, - I'm using the fitting procedure from
lmfit.Model
to find the parameters. (why lmfit? The rest of our processing is already in python, and I couldn't get the statsmodels package to handle general model specifications with parameters; but identifying the statistical method is the primary reason for asking)
EDIT: additional context as requested in comment
- Overall aim: the dynamics of the network are for each node in the network to settle into a decision. The individuals can choose from one of two states, and we are interested in the conditions under which:
- a) the fraction of nodes in network reaching the same state is maximised, and
- b) how fast this happens.
Form of data: If we label the control experimental group #1 and the treatment group #2, the data looks like:
# time, average fraction of nodes with majority choice, experimental group
60, 0.52, 2
70, 0.53, 2
80, 0.55, 2
90, 0.54, 2
...
1200, 0.92, 2
The "average fractions to majority choice" are averages across the replicates carried out.
We have 15 replicates or more per treatment, and the analysis is on data already obtained.
group #1 has lower average fractions by the end of the experiment (because they don't have the same level of agreement in general).
The curve following a saturation equation (above) fits the data reasonably well and fits our hypotheses for the underlying dynamics. So we are not looking to select between models.
However, we do want to say whether the `treatment' makes a difference.
So, the question here is whether difference between the fitted curves is statistically significant. What is the most appropriate way to test this?