Statistical comparison of numerous nonlinear model parameters

Question

I have 84 data sets (n=3) corresponding to 28 conditions (sample composition and temperature) and have fit my data set to the following nonlinear model using MATLAB nonlinear curve fitting:

$$y = \bigg[\frac{A}{1+\exp(-B\cdot(x-C))}\bigg]+D\cdot x$$

where the $y$ is mass [g] and $x$ is time [s].

How can I statistically and simultaneously compare my model parameters across all my conditions (i.e. A from condition 1 is statistically different from condition 2)? My first thought was to do an ANOVA+Tukey, but I do not know if this is valid with a nonlinear model parameter as the response.

Answer 1

There is no inherent problem with using an ANOVA-type approach and appropriate correction for multiple comparison, like Tukey's, with a nonlinear model. Your response is the outcome mass $y$, not the parameter values themselves. Standard statistical testing on a nonlinear fit depends on normality of the residuals about the predicted response values of $y$ and normality of the sampling distributions of the estimates of the parameter values.

The approach is nicely implemented in the R nls() function, as explained for example in this answer to a related question; scroll down to the part on "Comparing fits to different data sets."

The idea is to include each of the 28 conditions as an indexing variable. As with standard ANOVA, you first compare a model ignoring the specific conditions (with only 4 parameter values to be estimated) against a model that allows for the parameters to differ among the conditions (with 28*4 = 112 parameter values estimated). If a significance test shows the the reduction in residual sum of squares adequately balances off the need to estimate an extra 108 parameter values, then you can conclude that some conditions have different parameter values than others and proceed to more detailed examination.

If you think about ANOVA as being equivalent to a linear regression model with categorical predictors, this makes sense. The comparisons among conditions in ANOVA are comparisons among the coefficient estimates associated with the conditions, with sampling distributions that tend to be normally distributed in the limit of a large data set. If you are willing, in your non-linear model, to assume normal distributions of those sampling distributions then you can apply Z-tests, with correction for multiple comparisons, to evaluate differences among conditions.

One potentially big problem I see is in initializing the models in a way that works for all the conditions. With a 4-parameter nonlinear model and 28 different conditions that could be quite a challenge. A good deal of looking at the data first would seem to be warranted, to see if there are natural groupings that could reduce this to a few sets of related conditions (e.g., grouping by species, by condition, by temperature), as both the initial ANOVA test and the subsequent multiple comparison corrections will tend to lose a lot of power with so many conditions evaluated at once.

Finally, if this is for establishing conditions for your own work rather than for publication, you can consider a tradeoff in terms of "statistical significance." The classic p < 0.05 cutoff is a protection against mistakenly assuming a significant difference when there is none. For establishing adequate working conditions you might be more interested in practically significant differences than in statistically significant differences, using the results of the analysis as a guide to how big a risk you are taking by not finding "the best" conditions.