I would like to evaluate models based on their goodness of fit with data. Each model produces predicted values that should be correlated to the data but are not necessarily scaled the same way. So far I have compared different models by looking at the correlation between predicted values and the data but correlations do not take the model complexity (number of free parameters) into account. I know that AIC and BIC are common measures of goodness of fit that account for model complexity but these measures are based on the likelihood of the data given the model or the sum of squared residuals. I am not sure how to apply AIC and BIC in my case. How can I measure goodness of fit while taking model complexity into account in my case?

I am using R (in case it matters).

Many thanks!

  • $\begingroup$ I believe that you cannot use AIC or BIC if the dependent variable values are not the same. This can occur if you have some observations dropped for missing values in some models and not others; if you transform the dependent variable; or if a generalized linear model is used that changes the dependent variable. As you mention, you could compare predicted values to actual values, but they would have to be on the same scale as the original data. There are some measures like this in R here: rcompanion.org/handbook/G_14.html. $\endgroup$ – Sal Mangiafico Jul 5 '17 at 13:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.