Skip to main content
added 21 characters in body
Source Link
Florian Hartig
  • 8.7k
  • 30
  • 52

There are dozens of methods that have been proposed. We discuss and evaluate them in some length in

Dormann, et al. Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference. Ecological Monographs 88, no. 4 (2018): 485-504.

Some specific comments to your question:

  • If you give all the weight to the regression with the smallest AICc, this is AIC model selection. We discuss in the Dormann et al. that and why model averaging often outperforms model selection.
  • About how to weight the average: AIC is an option, but it is relatively aggressive (meaning it puts a lot of weight on few models). Tactical approaches that split weights more evenly (e.g. via CV error ) or even weights often outperformed exp(-AIC) weights. I think theThe underlying reason is that the exp(-AIC) is basically usedacts like a Posterior model weight, so it'slike a modified exp(-BIC) weight, and this makes only sense if the true model is in the set (M-closed). I know that B&A state that AIC weights can be used also if the true model is not in the set, but the point is that if all models in the set are biased, and you put too much weight on one model, you can't improve over this bias, while an average of the models can (and AIC will tend to put all weight on one model asymptotically).

There are dozens of methods that have been proposed. We discuss and evaluate them in some length in

Dormann, et al. Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference. Ecological Monographs 88, no. 4 (2018): 485-504.

Some specific comments to your question:

  • If you give all the weight to the regression with the smallest AICc, this is AIC model selection. We discuss in the Dormann et al. that and why model averaging often outperforms model selection.
  • About how to weight the average: AIC is an option, but it is relatively aggressive (meaning it puts a lot of weight on few models). Tactical approaches that split weights more evenly (e.g. via CV error ) or even weights often outperformed exp(-AIC) weights. I think the underlying reason is that the exp(-AIC) is basically used like a Posterior model weight, so it's a modified exp(-BIC) weight, and this makes only sense if the true model is in the set (M-closed). I know that B&A state that AIC weights can be used also if the true model is not in the set, but the point is that if all models in the set are biased, and you put too much weight on one model, you can't improve over this bias (and AIC will tend to put all weight on one model asymptotically).

There are dozens of methods that have been proposed. We discuss and evaluate them in some length in

Dormann, et al. Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference. Ecological Monographs 88, no. 4 (2018): 485-504.

Some specific comments to your question:

  • If you give all the weight to the regression with the smallest AICc, this is AIC model selection. We discuss in the Dormann et al. that and why model averaging often outperforms model selection.
  • About how to weight the average: AIC is an option, but it is relatively aggressive (meaning it puts a lot of weight on few models). Tactical approaches that split weights more evenly (e.g. via CV error ) or even weights often outperformed exp(-AIC) weights. The underlying reason is that exp(-AIC) basically acts like a Posterior model weight, so like a modified exp(-BIC) weight, and this makes only sense if the true model is in the set (M-closed). I know that B&A state that AIC weights can be used also if the true model is not in the set, but the point is that if all models in the set are biased, and you put too much weight on one model, you can't improve over this bias, while an average of the models can (and AIC will tend to put all weight on one model asymptotically).
Source Link
Florian Hartig
  • 8.7k
  • 30
  • 52

There are dozens of methods that have been proposed. We discuss and evaluate them in some length in

Dormann, et al. Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference. Ecological Monographs 88, no. 4 (2018): 485-504.

Some specific comments to your question:

  • If you give all the weight to the regression with the smallest AICc, this is AIC model selection. We discuss in the Dormann et al. that and why model averaging often outperforms model selection.
  • About how to weight the average: AIC is an option, but it is relatively aggressive (meaning it puts a lot of weight on few models). Tactical approaches that split weights more evenly (e.g. via CV error ) or even weights often outperformed exp(-AIC) weights. I think the underlying reason is that the exp(-AIC) is basically used like a Posterior model weight, so it's a modified exp(-BIC) weight, and this makes only sense if the true model is in the set (M-closed). I know that B&A state that AIC weights can be used also if the true model is not in the set, but the point is that if all models in the set are biased, and you put too much weight on one model, you can't improve over this bias (and AIC will tend to put all weight on one model asymptotically).