3
$\begingroup$

Let's say I have an experiment which yields discrete results between 1 and $N$. I am modelling the results using a number of statistical models and want to use Akaike (corrected) or Bayesian Information Criterion to choose the best model. How can I derive AICc or BIC if the predicted variables are discrete and bounded? Is there a difference in what we mean by "sample size" in AICc or BIC formula?

$\endgroup$
4
  • 1
    $\begingroup$ I believe AICc is derived for linear models anyway, so you'd be guessing in that case anyway. I would probably go back to the original BIC derivation and see where the $\log(n)$ term emerges from/how it would be modified for discrete/low-information responses. $\endgroup$
    – Ben Bolker
    Commented Sep 20, 2015 at 1:24
  • $\begingroup$ In Claeskens and Hjort, "Model Selection and Model Averaging", n is the number of "observed data". What is "observed data"? If I am conducting a presidential election Poll and ask 1000 people "Will you vote for Bush or for Gore", than is n=1000 (because I got 1000 binary answers) or is n=1 (because I've got one independent data point: the fraction of the interviewees who will for vote for Bush)? $\endgroup$
    – quant_dev
    Commented Sep 20, 2015 at 12:18
  • 1
    $\begingroup$ n=1000. Check this thread and this thread for when it is OK to use AIC for model comparison. I mostly agree with the first thread but am suspicious of the second one. $\endgroup$ Commented Oct 19, 2015 at 18:11
  • $\begingroup$ I am shocked that there is so much ambiguity about this issues. Isn't statistics a part of mathematics? Shouldn't there be a clear proof when X works and when it doesn't? $\endgroup$
    – quant_dev
    Commented Oct 21, 2015 at 9:01

1 Answer 1

1
$\begingroup$

"I am shocked that there is so much ambiguity about this issues. Isn't statistics a part of mathematics? Shouldn't there be a clear proof when X works and when it doesn't?"

My answer: one of the most important things I ever learned was that while statistics does heavily involve math, it is not just math. It shares a lot with fields like law and politics. Some of the most important questions in statistics come down to value judgments: What's your goal? What's your data? What are you willing to assume? What can you get out of a model that is very likely to be much simpler than our incredibly complex reality.

But, yes, it would be nice to have a clear answer to the question, "When my data are discrete (in my case, binary or categorical), is the traditional AICc appropriate?"

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.