Using AIC for to select models with equal number of parameters

Question

My research project is looking at various measurements that might predict wheat yield better than tiller (stem) counts. Some of my site-years have a measurement that requires a polynomial regression for best fit to yield. The nature of these measurements (counts, sensor readings, and percentages) create a huge difference in variation around the regression line- and I want a more "universal" statistic to compare them to against tiller counts.

So here is the question... can/should I use AIC to compare these models, most of which only have 1 parameter? Please note, that I am ONLY comparing within site-years, meaning the dependent variables are identical between each model being compared- the only difference is the independent variable of the model. Might adjusted R2 be a more appropriate statistic in this case?

I'm a statistics novice and really need help.... Thanks!

Answer 1

You can use AIC in your situation.

However, if your aim is explicitly prediction, I'd say a measure of out-of-sample predictive accuracy would be more intuitive. Potentially including cross-validation. Just choose a good error measure or loss function, possibly mse, or something else. You could also look at the diebold-mariano test if you want to assess whether one model's predictions are significantly better than another one's.