# Log-normal distribution versus chi-square distribution for comparing RMSE of nonlinear fits

I have parameter estimates fitting a particular nonlinear model for thousands of experimental cycles. My goal is to find a nice way to tag those cycles which didn't fit the model very well. Currently I am comparing RMSE values for this. I have a huge list of RMSE values and am looking for a cutoff RMSE past which to flag it.

I plotted the histogram of RMSEs and found that they fit a log-normal distribution. However, I was informed that the proper distribution to use in this case is $\chi^2$. For my purposes the distribution I use doesn't seem to matter as long as it fits well, but since I was curious, I fitted a $\gamma$ distribution (which wikipedia informed me $\chi^2$ was a special case) to the RMSE histogram, but the parameters didn't seem to fit quite as well. Is the $\gamma$ distribution really that more meaningful?

Also, if you have any alternative suggestions for approaching my problem I would love to hear them.

-
Could you explain how a single number, the RMSE, can "fit" a distribution? Are you trying to say that the residuals are well approximated by a lognormal distribution? Or perhaps that you get a lower RMSE when using a regression procedure that assumes a lognormal distribution of residuals? Or maybe something else? –  whuber Dec 19 '11 at 18:43
Is my question clear now? –  wdkrnls Dec 19 '11 at 19:31
It's better, but I still find parts of it confusing. Are you saying that each "experimental cycle" produces a set of data, you perform nonlinear regression on those data, and compute an RMSE for that regression? You thereby get one RMSE per "cycle" and find that this collection of RMSE's appears lognormal. Did you determine that solely through visual examination of the histogram? How did you fit the gamma distribution? BTW, the reason to suppose you should get a gamma is weak: it is based on several assumptions that often are not true, especially not for nonlinear models. –  whuber Dec 19 '11 at 20:51
Your description of what I did is correct. I'm treating each "cycle" as a separate data set and taking regressions of each. And yes, I determined the data fit a log-linear distribution solely through visual examination. However, the Kolmogorov D test gives a D value less than the critical value. JMP fit the gamma distribution automagically. –  wdkrnls Dec 19 '11 at 23:32