I have imaging data where I imaged different instances of the (somewhat) same phenomenon (2 different experimental conditions). I have already settled on the equations I use for curve fitting and which I would like to compare:
- error function
- error function + a gaussian
- error function + an exponential decay convolved with a gaussian
For the latter two, I have 4 variants each, differing in their degrees of freedom (e.g. the error function and the gaussian may be shifted along x in one variant, in the other variant the shift is set to 0). Another degree of freedom is whether I allow the error function and the gaussian to have different sigmas. Thus, in total I have 9 different equations (see attached picture of one signal (in yellow) and the 9 equations fitted to the data by minimizing the Residual sum of squares). I guess the proper way to say this is that I have 9 models which I'd like to compare.
I have two questions:
How can I compare the sensitivity between the two conditions to the various degrees of freedom? For example, does allowing for shift along x improve fits of condition-1-data significantly more than fits of condition-2-data? What I did so far was to compare relative changes in the median value of the Rsquared values of all curves (per condition). Might this be flawed? Perhaps another way would be to use changes in some measure of error, e.g. the median of the relative changes in sums of squared errors? Or something else entirely?
How can I find out which degrees of freedom significantly improve my fits? I looked into information criteria and adjusted r squared and so forth, but I don't understand how I could apply any of those to my data, and whether that would even be possible
It may of course be that my whole approach of fitting individual curves is flawed. But I don't see a way of otherwise making sense of my data. The other thing I thought about was to overlay all datapoints, but the problem is that the data differs in terms of amplitude and width (e.g. sigma of the gaussian), and aligning along x seems difficult too when allowing for a shift along x between error function and gaussian/exponential. The tags I gave this topic reflect the things I have looked into so far, but I have to confess I don't really know anything about any of those (except curve fitting, and perhaps statistical significance)
Perhaps it is worth noting that my dataset is ~400 instances per condition