Use BIC or AIC as approximation for Bayesian Model Averaging [duplicate]

Question

I want to compare "real" Bayesian Model Averaging (BMA) performed with the EM algorithm and information-criterion based BMA.

Which one, BIC or AIC, is a "closer" approximation to the "real" BMA?

BIC as the name is indicating? I would like to compare apples with apples.

My models are not nested with 1 parameter at minimum and 3 parameter at maximum.

Answer 1

I don't understand the first sentence very well, but if your asking whether you should use BIC or AIC to approximate BMA than the answer is fairly straightforward.

You would use BIC rather than AIC to approximate posterior model probabilities which can then be used as weights in BMA.

Given a set of models $M_1,M_2,…,M_n$ and respective BIC scores $B_1,B_2,…,B_n$, the posterior probability for model $j \in \{1,2,...,n\}$ can be approximated as; $$ Pr(M_j|D) \approx \frac{\exp\bigg(\frac{\mathrm{B_j}}{-2}\bigg)}{\sum_{i=1}^n \exp\bigg(\frac{\mathrm{B_i}}{-2}\bigg)} $$ Where $D$ is "the data". I briefly touch on why this is the case here. You can also see a good reference here, or take a look at the original paper where BIC is derived (Schwarz, 1978) which inevitably leads to this result.

The posterior probabilities $Pr(M_j|D)$ are used as weights in BMA, and it should be obvious that $\sum_{j=1}^n Pr(M_j|D) = 1$.

If you plan to do this in real life, I would advise using Bayes factors to avoid computational difficulties (overflow/underflow). In this case you would choose one BIC value, say the lowest one, which we will denote as $B_{*}$ then letting $\Delta_i=B_i-B_{*}$ calculate;

$$ Pr(M_j|D) \approx \frac{\exp\bigg(\frac{\Delta_j}{-2}\bigg)}{\sum_{i=1}^n \exp\bigg(\frac{\Delta_i}{-2}\bigg)} $$

This helps avoid issues of getting extremely large/small positive numbers when exponentiating the BIC values.

• Schwarz, G. (1978). Estimating the dimension of a model. The annals of statistics, 6(2), 461-464.