0
$\begingroup$

I would like to compare two models using the BIC and AICc. Doing so seems fairly straightforward if both models are fit to only one dataset. However, I have data from 10 participants, and there is no reason to assume that they are all well-described by the same set of parameters. Is it appropriate to sum the log-likelihoods of a model across all participants and multiply the number of parameters by the number of participants to obtain a "group BIC/AICc" for that model, or is there another way of obtaining the BIC/AICc in such a situation?

$\endgroup$
1
$\begingroup$

I did not see the rationale of using your "group BIC/AIC". Your proposed group BIC/AIC is comparing a different pair of models. Let $f(\mathbf{x}_i \mid \alpha)$ and $g(\mathbf{y}_i \mid \beta)$ be two competing models. For simplicity, suppose we have two groups, ande denote $D_i$ the group dummy variable. $D_i=1$ indicates that $i$ belongs to group $1$, and $D_i=0$ refers to the case that $i$ belongs to group $2$. Correspondingly, let $\alpha_1$ and $\alpha_2$ be the two sets of parameters specifying the distribution of group $1$ and $2$, respectively. $\beta_1$ and $\beta_2$ are defined similarly. Define two new densities/models: $$\tilde{f}(\mathbf{x}_i, D_i \mid \alpha_1, \alpha_2)=f(\mathbf{x}_i\mid \alpha_1)^{D_i}f(\mathbf{x}_i\mid \alpha_2)^{1-D_i},$$ $$\tilde{g}(\mathbf{y}_i, D_i \mid \beta_1, \beta_2)=g(\mathbf{y}_i\mid \beta_1)^{D_i}g(\mathbf{y}_i\mid \beta_2)^{1-D_i}.$$ Note that $$\log\tilde{f}(\mathbf{x}_i, D_i \mid \alpha_1, \alpha_2)=D_i \log f(\mathbf{x}_i\mid \alpha_1)+(1-D_i)\log f(\mathbf{x}_i\mid \alpha_2),$$ $$\log\tilde{g}(\mathbf{y}_i, D_i \mid \beta_1, \beta_2)=D_i \log g(\mathbf{y}_i\mid \beta_1)+(1-D_i)\log g(\mathbf{y}_i\mid \beta_2).$$ If you fit the above two models to the two groups of data, the ordinary AIC/BIC would be your AIC/BIC. I am not sure if this connection might help you.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks. I think it's equivalent to what I am doing, but conceptualized better :) $\endgroup$ – Pavel Jul 10 '13 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.