2
$\begingroup$

I am collecting a lot of data from neurons and will be analyzing each neuron separately. Is it valid to pick the best of several models based on the AICc for each neuron and then use the ANOVA p-values from the best model? Or should should I only run the AICc on preliminary data and then use the best model analyze every neuron?

My impression was that it is only statistically valid to select the model in the preliminary experimental design phase, but I'm now having difficulty finding this explicitly stated. References would be greatly appreciated.

$\endgroup$
3
$\begingroup$

I believe I have found my own answer. It sounds like using AIC to select from a large set of conceivable variables can result in models that are "excessively" tailored to the data at hand, ie. data dredging. However, if selecting from a limited set of a priori models, then it should be fine.

Reference:

Question 4 from Joseph E. Cavanaugh's slides at: http://myweb.uiowa.edu/cavaaugh/ms_lec_14_ho.pdf

$\endgroup$
2
$\begingroup$

Here are a couple of good papers and books that discuss the use of AIC in the ecological disciplines. They emphasize the importance of a "little hard thinking" or "navel gazing" to develop an a priori model set.

Anderson, D. R., and K. P. Burnham. 2002. Avoiding pitfalls when using information-theoretic methods. Journal of Wildlife Management 66(3):912-918.

Anderson, D. R., K. P. Burnham, W. R. Gould, and S. Cherry. 2001. Concerns about finding effects that are actually spurious. Wildlife Society Bulletin 29(1):311-316.

Burnham, K. P., and D. R. Anderson. 2002. Model selection and multimodel inference: a practical information-theoretic approach. Second edition. Springer, New York, USA.

Anderson, D. R. 2008. Model based inference in the life sciences: a primer on evidence. Springer, New York, USA.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.