# Questions tagged [model-averaging]

The process of combining different models to get a better resulting model than any of the constituents. Eg, computing a parameter estimator as the average of the estimators from each component model.

95 questions
Filter by
Sorted by
Tagged with
1 vote
12 views

### GLMM Model Averaging with Predictor Multicollinearity

I am running GLMM models to determine how environmental factors influence bird collisions. I've obtained a list of candidate models with delta AIC less than 2, and I want to perform model averaging. I ...
• 11
1 vote
21 views

### How to calculate $P(f_1(X) = \text{max}(f_1(X), \dots, f_K(X))$ when $X$ is multivariate Normal?

Let's say I have a multivariate distribution $\mathbf{X} \sim \text{MVN}(\mathbf{\mu}, \mathbf{\Sigma})$ and a set of $K$ scalar functions of $\mathbf{X}$, $f_1(\mathbf{X}), \dots, f_K(\mathbf{X})$. ...
• 35k
24 views

### Computing Bayesian model averaged posteriors

The Bayesian model averaged posterior predictive distribution for new data $\tilde{y}$ given training data $y$, across a set of $M$ models $\mathcal{D} = \{D_{1}, ..., D_{M}\}$, is defined as: \begin{...
• 185
22 views

### Interpreting Contradictory Results in Bayesian Model Averaging: High Posterior Inclusion Probability with Unclear Effect

In my research, I am utilizing the Bayesian Model Averaging (BMA) methodology to identify the best set of regressors that can predict the outcome variable $y$. My dataset consists of five variables ...
• 37
9 views

I have a large number of MARSS dynamic factor analysis models that, based on AICc, are all competitive for the 'best' model. Is there a way to implement a model averaging process so that I can extract ...
• 283
37 views

### BMA formula with BIC

I am interested in using Bayesian modele averaging as a selection creteria (BMA) vs AIC. I read that BMA is widely implemented in clustering models. Suppose that we need to fit M models to a data and ...
• 650
1 vote
66 views

### Cross validation with GLMMs; best way to partition train and test data with regard to random effects?

I have some analysis I'm working on and I'm having a hard time nailing down the correct approach to take. I am modeling the dynamics of frog choruses by looking at what predicts the outcome of calling ...
1 vote
44 views

### Help with Bayesian model averaging

I am new to statistical modelling and I need some help with Bayesian model averaging. I have 3 models and I would like to derive a BMA of these models. I am using the BIC estimate for the different ...
• 11
1 vote
21 views

### General question on model weighting and averaging

I've had a stats related question I've been wondering for a while, but for which I have yet to find good sources on. I'm aware of things like the Akaike Information Criterion for weighting candidate ...
• 11
1 vote
128 views

### Weird AIC weights in model averaging

I conducted AIC selection on a set of models, and isolated my top 3 models. I then calculated the AIC weights of each model, and got the values 0.99, 1.92e-14,and 6.9e-18. I never saw weight values ...
• 151
1 vote
23 views

### Is it possible to average models with different link functions?

While I can compare models that have different link functions in terms of AIC/BIC weights, I think it's impossible to use those weights to create an averaged model. Am I right in believing this?
• 1,109
182 views

### Bayesian model averaging

In which situation should one refrain from using BMA? It seems to me that it is always a good idea to use the posterior probabilities when inferring/predicting.
• 53
506 views

### When/what to standardize for model-averaging with(out) interactions

I’m using the {MuMIn} package in R to select models (dredge, get top models, average etc). My question is about whether I need to, or should, standardise my ...
889 views

### Averaging SVM and GLM results: sensible or stupid?

I have taken two different approaches to calculate probability: using a GLM and an SVM. They are giving slightly different results (which is understandable, they are completely different approaches). ...
1 vote
36 views

### BMA and RJMCMC predictive performance

We have a family of statistical models, with parameter spaces of different dimensions, which we aggregated through standard Bayesian Model Averaging (BMA). Experiments using training and test sets ...
• 24.6k
285 views

### Does bagging work for OLS to improve prediction?

In Elements of Statistical Learning, section 8.7, the author states that The bagged estimate will differ from original estimate only when the latter is a nonlinear or adaptive function of the data I ...
• 321
2k views

### In what applications do we prefer Model Selection over Model Averaging?

I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
• 2,966
209 views

### Is post-variable-selection multimodel inference a bad idea?

If I understood correctly, in this answer, Ben Bolker says that using inferential methods after having performed AIC-based model selection is wrong because "standard inferential methods assume ...
• 641
974 views

### Do "conditional" averaged coefficients *ever* make sense?

This question is related to one on Stack Overflow where the user wanted to obtain predictions and their standard errors using the "conditional" averaged coefficients from the R package MuMIn....
• 20.7k
353 views

### Is model averaging possible/appropriate with a GAM?

I'm new here - apologies if there are any problems. I have two GAMs that I've fit (using mgcv::gam()), and the two models are comparable in AICc values. With GLMs, ...
• 21
366 views

### Can Bayesian Model Averaging be Optimal when the Hypothesis Space does not contain the true hypothesis?

I am utterly confused. I have been reading about the optimality of Bayes classifier and Bayesian model averaging all the time, but when I try to dig deeper, I just get more confused. On the one hand, ...
• 748
1 vote
105 views

### How can we assume the models are exhaustive in Bayesian Model Averaging?

Bayesian model averaging is justified using the law of total probability which requires the the set of models that we average over to be exhaustive. Shouldn’t we prove that the set of models are ...
1 vote
138 views

### Model averaging parameter estimates for glmer models using AICmodavg in R

I encounter the following error trying to calculate model-averaged parameter estimates across a set of GLMM models using the AICmodavg package version 2.3.1. in RStudio(version1.2.5) ERROR: Error in ...
1 vote
336 views

### When is it acceptable to compute (conditional) subset-averaged coefficients?

I'm running an ecological study and I have 4 dependent variables (DVs) that I would like to explain (my interest thus lies in inference and not in prediction). For each one of these variables, I built ...
• 641
159 views

### Model averaging a GLM function with equal weights

Background Let me first say that I read this post and that I looked at the BMS vignette. I used the package sure (CRAN, R ...
• 528
102 views

### Bayesian Model Averaging (Bayesian Averaging of Classical Estimates) Issue

I am trying to implement a method used by this paper (Described briefly at the bottom of page 3): From what I understand, it does 2$^k$ OLS regressions each time step to forecast the next time step ...
• 21
67 views

### Is it a good idea to use a linear model (like logistic regression) to generate new features for a non linear model (like random forest)? [duplicate]

The setting is a 2-class classification problem. We have too many features, some of them not very informative and with many zeros. We are thinking in ways of selecting the best features, and PCA (in ...
572 views

### AIC model averaging when models are correlated

AIC model-averaging: In "standard" AIC model averaging we average models with weights proportional to $$w_i \propto \exp( -0.5 \times \Delta \text{AIC}_i ),$$ where $\Delta \text{AIC}_i$ is ...
• 33.5k
1 vote
61 views

### Using AIC weights to determine prediction intervals for a single model structure

I am working with fitting regression models to data, and producing prediction intervals would be useful. Unfortunately, the data often has few data points, and is reported as mean rather than ...
• 41
3k views

### Model averaging: predictor significance vs importance

What is the correct way to report the results of model averaging? I’m using MuMin in R, results include a p-value. It’s also possible to get the importance of each predictor. Example results below. ...
• 125
1 vote
56 views

### Bayesian model averaging when none of the models is well specified

Usually, from what I've read of Bayesian Model Averaging (BMA), a typical assumption is that one of the models is well specified... However, what will happen when all models are misspecified? What ...
186 views

### Why is pseudo-Bayesian Model Averaging using WAIC giving counterintuitive results for my model? Issue with unconstrained data uncertainty?

I'm currently working on a model which fits a pair of curves which are parameterized by some shape and scaling parameters. I want to produce weights for two different models. In my first model, which ...
• 21
270 views

### Regularization via model averaging?

Say you have the model $$\Phi^{-1} \left(y\right)=\beta_0 + \beta_1 x$$ I am interested in adding some regularization, specifically concerning the parameter $\beta_1$, to introduce some "skepticism" ...
• 3,273
1 vote
409 views

### Is It Valid To Calculate Model-Averaged Confidence Intervals In the Same Way As Model Averaged Predictions?

I'm fitting a series of mixed-effects models, and I'm trying to calculate the model-averaged predictions and their confidence intervals. If I have a set of $R$ models $\{M_1,...,M_R\}$ I know that ...
• 113
211 views

### When averaging models, which models need to meet assumptions?

I have 16 variables and am running all possible models (65,535 total!), then averaging the best models. A model including all variables has normally distributed residuals, but some of the 65,535 ...
1 vote
663 views

### Is there a difference between averaging individual regressions and including a random effect?

I have a bit of a theoretical question about random effects models and regression. If I have a set of clustered, longitudinal data (say repeated measurements of $y$ on a number of different ...
• 297
1 vote
164 views

### Combining regression models from separate data sets

What is the best way to combine regression betas from separate data sets? For example, a data set is split in two based on some fundamental characteristic, and the same two factor regression is run ...
1k views

### Weighting of multiple linear regressions in an ensemble

If I have a continuous dependent variable and N continuous predictors, and I fit all possible regressions with zero up to N variables, how should I weight those regressions for prediction? One ...
• 606
1 vote
104 views

### Is this Bayesian model averaging?

A classical example of Bayesian model averaging (BMA) is the regression setup where the choice of different sets of covariates corresponds to different models $\mathcal{M}_k$, $k = 1, \ldots, K$, ...
• 405
1 vote
284 views

### Can model averaging be applied on models fitted to different data sets?

I have two data sets collected from two different sets of participants on their behaviour. EDIT: Both have the same response variables (Propensity to behave in a certain way - Yes or No). But they ...
• 11
1 vote
200 views

### BMA - at what posterior effect probability can we say that a variable has an effect?

in the setting of a Bayesian model averaging, we are not dealing with P--value when we are assesing variable importance, but with the posterior effect probability of each variable. My question is, at ...
• 125
184 views

### Interpretation of a classic multinomial logit vs. BMA of multinomial logit

EDIT #1 Most likely I have set up the function bic.mlogit in a wrong way. @Jesper Hybel, hopefully, directed me in the right way. With the new setup I get two sets ...
• 125
87 views

### vector of weights

I am using the MuMIn package for model averaging. However, I am not clear of the function par.avg(). In this function, we need to specify the following, ...
• 81
1k views

### Effect sizes for model averaging

The goal is to find if one factor is stronger than the other in the models I have considered. I am using the information-theoretic approach. Since $n/K>40$, I am using AIC. Firstly the model is ...
• 81
2k views

### For model-averaging a GLM, do we average the predictions on the link or response scale?

To compute the model-averaged predictions on the response scale of a GLM, which is "correct" and why? Compute the model averaged prediction on the link scale and then back-transform to the response ...
• 646
1 vote
236 views

### Model Averaging errors using lme4 in R

I am running a glmer model on a response variable with binomial distribution and random term. My data has 3 explanatory categorical variables and I have successfully run dredge() on them and their ...
• 11
2k views

### Model averaging in mixed models

I am reading this blog post regarding using model averaging for mixed models and the last para says "https://sites.google.com/site/rforfishandwildlifegrads/home/mumin_usage_examples" ...
• 981
364 views

### Model-averaged for a glm.nb [closed]

How can i do a model averaged for a negative binomial model. I try with the AICcmodavg but it is not compatible with glm.nb models. Any help would be appreciated Thank you very much Magdalena
1 vote
573 views

### How does model averaging with a categorical variable work?

I have a series of models (~14) which do not include a categorical variable. One model (#15) however, does have a categorical variable with 3 levels. Normally for this model one of the categories ...
• 21