2
$\begingroup$

I have a number of distributions which I want to fit to a CDF that is comprised of one or more Gaussian CDFs. I was able to use weighted least squares regression to find the best fit parameters for each regression depending on the number of modes I give the model (e.g. each Gaussian CDF has three parameters, mu, sigma, and a scaling factor, that are summed to create my total function). In addition, it is reasonable that the CDF may not actually approach 1. If the fit places the CDF between approximately 0.8 and 1.2, it would be physically justifiable. I've included example plots of my 1, 2, and 3 mode fits for a distribution (Ignore scale on left of plots). enter image description here

I am trying to figure out if there is a good way to determine if one model is better than another without overfitting. My thought was to try using AIC, but I'm not sure the methods I'm using are the correct way of thinking about this problem or not.

Currently, I am minimizing the weighted least squares by minimizing: $$ \sum_i [w_i * (f(x_i) - y_i)^2] $$ Where $w_i$ is the weight for each datapoint. I'm not getting confused though, as what I'm finding seems to indicate that log-likelihood would be something like: $$ log L = \sum_i \frac{w_i * (f(x_i) - y_i)^2}{2\sigma^2} $$ Is this actually the log-likelihood? Why is it not being maximized? And additionally, I'm trying to understand how this fits in to AIC or AICc in order to penalize the additional parameters with the multiple mode fits. Does anyone have suggested resources for better understanding model selection for nonlinear regression? Most of the information I have found has been on Maximum Likelihood Estimation for probability distributions rather than this sort of sigmoid curve fitting.

$\endgroup$
  • $\begingroup$ Note that if $\sigma^2$ doesn't vary across $i$, it won't affect the maximization, so the weighted least squares results would be the same as the log-likelihood results. $\endgroup$ – jbowman Mar 11 '14 at 20:45
2
$\begingroup$

AIC (or AICc) is commonly used for selection of the preferred model. This reference (http://www.mun.ca/biology/quant/ModelSelectionMultimodelInference.pdf) has some good background information.

The AIC = logL + 2*k (where k=number of parameters). So, the AIC maximizes the log-likelihood but then penalizes for each additional dimension (aka parameter) that is available to the problem. A simple way to think about it (not quite correct, but it shows the message) is that with each parameter, you add a dimension for fitting, and in that dimension you can go two directions relative to your starting point (up or down).

The AIC collapses to a chi-squared test with k degrees of freedom when you add one parameter to a nested model with a p-value of 0.15.

Sometimes variants on the AIC are used that include a different per-parameter penalty than 2 (my preference is 6.63 which equates to the chi-squared test with one degree of freedom and p=0.01).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.