Questions tagged [model-averaging]
The process of combining different models to get a better resulting model than any of the constituents. Eg, computing a parameter estimator as the average of the estimators from each component model.
93
questions
0
votes
0
answers
32
views
BMA formula with BIC
I am interested in using Bayesian modele averaging as a selection creteria (BMA) vs AIC. I read that BMA is widely implemented in clustering models.
Suppose that we need to fit M models to a data and ...
- 640
1
vote
0
answers
38
views
Cross validation with GLMMs; best way to partition train and test data with regard to random effects?
I have some analysis I'm working on and I'm having a hard time nailing down the correct approach to take. I am modeling the dynamics of frog choruses by looking at what predicts the outcome of calling ...
- 11
1
vote
0
answers
38
views
Help with Bayesian model averaging
I am new to statistical modelling and I need some help with Bayesian model averaging.
I have 3 models and I would like to derive a BMA of these models.
I am using the BIC estimate for the different ...
- 11
1
vote
0
answers
16
views
General question on model weighting and averaging
I've had a stats related question I've been wondering for a while, but for which I have yet to find good sources on. I'm aware of things like the Akaike Information Criterion for weighting candidate ...
- 11
0
votes
0
answers
27
views
Stuck on interpretation/validation of GLMM results
I am using glmmTMB in R to identify which weather variables (n=5) most influence annual bird counts (n=5 responses) from different monitoring sites. So far, I have ...
- 33
0
votes
0
answers
85
views
Inclusion of candidates in model averaging with GLMMs
I am using GLMMs to examine the influence of 5 weather variables on different biological count variables in R. With n=5 weather variables and modelling main effects only, I have 32 candidate models (...
- 33
0
votes
0
answers
14
views
Is it necessary to standardize predictors when model averaging without interactions?
I am using GLMMs to model the influence of weather variables on bird counts, and using model averaging to generate parameter estimates for each predictor. Standardizing (centralizing) predictors is ...
- 33
1
vote
1
answer
100
views
Weird AIC weights in model averaging
I conducted AIC selection on a set of models, and isolated my top 3 models. I then calculated the AIC weights of each model, and got the values 0.99, 1.92e-14,and 6.9e-18. I never saw weight values ...
- 151
1
vote
0
answers
20
views
Is it possible to average models with different link functions?
While I can compare models that have different link functions in terms of AIC/BIC weights, I think it's impossible to use those weights to create an averaged model. Am I right in believing this?
- 1,170
4
votes
1
answer
128
views
Bayesian model averaging
In which situation would you refrain from using BMA. It seems to me it is always a good idea to use the posterior probabilities when infering/predicting.
- 43
0
votes
1
answer
385
views
When/what to standardize for model-averaging with(out) interactions
I’m using the {MuMIn} package in R to select models (dredge, get top models, average etc). My question is about whether I need to, or should, standardise my ...
- 1
10
votes
1
answer
874
views
Averaging SVM and GLM results: sensible or stupid?
I have taken two different approaches to calculate probability: using a GLM and an SVM. They are giving slightly different results (which is understandable, they are completely different approaches). ...
1
vote
0
answers
31
views
BMA and RJMCMC predictive performance
We have a family of statistical models, with parameter spaces of different dimensions, which we aggregated through standard Bayesian Model Averaging (BMA). Experiments using training and test sets ...
- 23.2k
2
votes
1
answer
231
views
Does bagging work for OLS to improve prediction?
In Elements of Statistical Learning, section 8.7, the author states that
The bagged estimate will differ from original estimate only when the latter is a nonlinear or adaptive function of the data
I ...
- 321
10
votes
4
answers
2k
views
In what applications do we prefer Model Selection over Model Averaging?
I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
- 2,526
5
votes
1
answer
167
views
Is post-variable-selection multimodel inference a bad idea?
If I understood correctly, in this answer, Ben Bolker says that using inferential methods after having performed AIC-based model selection is wrong because "standard inferential methods assume ...
- 611
5
votes
2
answers
732
views
Do "conditional" averaged coefficients *ever* make sense?
This question is related to one on Stack Overflow where the user wanted to obtain predictions and their standard errors using the "conditional" averaged coefficients from the R package MuMIn....
- 19.4k
2
votes
0
answers
297
views
Is model averaging possible/appropriate with a GAM?
I'm new here - apologies if there are any problems.
I have two GAMs that I've fit (using mgcv::gam()), and the two models are comparable in AICc values. With GLMs, ...
- 21
2
votes
1
answer
332
views
Can Bayesian Model Averaging be Optimal when the Hypothesis Space does not contain the true hypothesis?
I am utterly confused. I have been reading about the optimality of Bayes classifier and Bayesian model averaging all the time, but when I try to dig deeper, I just get more confused.
On the one hand,
...
- 728
1
vote
3
answers
101
views
How can we assume the models are exhaustive in Bayesian Model Averaging?
Bayesian model averaging is justified using the law of total probability which requires the the set of models that we average over to be exhaustive. Shouldn’t we prove that the set of models are ...
user297721
1
vote
0
answers
117
views
Model averaging parameter estimates for glmer models using AICmodavg in R
I encounter the following error trying to calculate model-averaged parameter estimates across a set of GLMM models using the AICmodavg package version 2.3.1. in RStudio(version1.2.5)
ERROR:
Error in ...
1
vote
0
answers
299
views
When is it acceptable to compute (conditional) subset-averaged coefficients?
I'm running an ecological study and I have 4 dependent variables (DVs) that I would like to explain (my interest thus lies in inference and not in prediction). For each one of these variables, I built ...
- 611
0
votes
0
answers
136
views
Model averaging a GLM function with equal weights
Background
Let me first say that I read this post and that I looked at the BMS vignette.
I used the package sure (CRAN, R ...
- 379
2
votes
0
answers
97
views
Bayesian Model Averaging (Bayesian Averaging of Classical Estimates) Issue
I am trying to implement a method used by this paper (Described briefly at the bottom of page 3):
From what I understand, it does 2$^k$ OLS regressions each time step to forecast the next time step ...
- 21
2
votes
1
answer
65
views
Is it a good idea to use a linear model (like logistic regression) to generate new features for a non linear model (like random forest)? [duplicate]
The setting is a 2-class classification problem. We have too many features, some of them not very informative and with many zeros. We are thinking in ways of selecting the best features, and PCA (in ...
9
votes
1
answer
493
views
AIC model averaging when models are correlated
AIC model-averaging: In "standard" AIC model averaging we average models with weights proportional to
$$w_i \propto \exp( -0.5 \times \Delta \text{AIC}_i ),$$
where $\Delta \text{AIC}_i$ is ...
- 30.5k
1
vote
0
answers
53
views
Using AIC weights to determine prediction intervals for a single model structure
I am working with fitting regression models to data, and producing prediction intervals would be useful. Unfortunately, the data often has few data points, and is reported as mean rather than ...
- 41
3
votes
1
answer
2k
views
Model averaging: predictor significance vs importance
What is the correct way to report the results of model averaging? I’m using MuMin in R, results include a p-value. It’s also possible to get the importance of each predictor. Example results below.
...
- 125
0
votes
0
answers
52
views
Bayesian model averaging when none of the models is well specified
Usually, from what I've read of Bayesian Model Averaging (BMA), a typical assumption is that one of the models is well specified...
However, what will happen when all models are misspecified? What ...
- 5,420
2
votes
0
answers
179
views
Why is pseudo-Bayesian Model Averaging using WAIC giving counterintuitive results for my model? Issue with unconstrained data uncertainty?
I'm currently working on a model which fits a pair of curves which are parameterized by some shape and scaling parameters. I want to produce weights for two different models. In my first model, which ...
- 21
5
votes
1
answer
251
views
Regularization via model averaging?
Say you have the model
$$ \Phi^{-1} \left(y\right)=\beta_0 + \beta_1 x$$
I am interested in adding some regularization, specifically concerning the parameter $\beta_1$, to introduce some "skepticism" ...
- 3,163
1
vote
0
answers
366
views
Is It Valid To Calculate Model-Averaged Confidence Intervals In the Same Way As Model Averaged Predictions?
I'm fitting a series of mixed-effects models, and I'm trying to calculate the model-averaged predictions and their confidence intervals. If I have a set of $R$ models $\{M_1,...,M_R\}$ I know that ...
- 113
0
votes
0
answers
184
views
When averaging models, which models need to meet assumptions?
I have 16 variables and am running all possible models (65,535 total!), then averaging the best models. A model including all variables has normally distributed residuals, but some of the 65,535 ...
1
vote
1
answer
598
views
Is there a difference between averaging individual regressions and including a random effect?
I have a bit of a theoretical question about random effects models and regression. If I have a set of clustered, longitudinal data (say repeated measurements of $y$ on a number of different ...
- 307
1
vote
0
answers
150
views
Combining regression models from separate data sets
What is the best way to combine regression betas from separate data sets?
For example, a data set is split in two based on some fundamental characteristic, and the same two factor regression is run ...
- 11
2
votes
2
answers
1k
views
Weighting of multiple linear regressions in an ensemble
If I have a continuous dependent variable and N continuous predictors, and I fit all possible regressions with zero up to N variables, how should I weight those regressions for prediction? One ...
- 596
1
vote
0
answers
99
views
Is this Bayesian model averaging?
A classical example of Bayesian model averaging (BMA) is the regression setup where the choice of different sets of covariates corresponds to different models $\mathcal{M}_k$, $k = 1, \ldots, K$, ...
- 405
1
vote
0
answers
264
views
Can model averaging be applied on models fitted to different data sets?
I have two data sets collected from two different sets of participants on their behaviour.
EDIT: Both have the same response variables (Propensity to behave in a certain way - Yes or No). But they ...
- 11
1
vote
1
answer
178
views
BMA - at what posterior effect probability can we say that a variable has an effect?
in the setting of a Bayesian model averaging, we are not dealing with P--value when we are assesing variable importance, but with the posterior effect probability of each variable. My question is, at ...
- 115
0
votes
1
answer
173
views
Interpretation of a classic multinomial logit vs. BMA of multinomial logit
EDIT #1
Most likely I have set up the function bic.mlogit in a wrong way. @Jesper Hybel, hopefully, directed me in the right way. With the new setup I get two sets ...
- 115
0
votes
1
answer
86
views
vector of weights
I am using the MuMIn package for model averaging. However, I am not clear of the function par.avg(). In this function, we need to specify the following,
...
- 81
3
votes
1
answer
1k
views
Effect sizes for model averaging
The goal is to find if one factor is stronger than the other in the models I have considered. I am using the information-theoretic approach. Since $n/K>40$, I am using AIC. Firstly the model is ...
- 81
14
votes
1
answer
2k
views
For model-averaging a GLM, do we average the predictions on the link or response scale?
To compute the model-averaged predictions on the response scale of a GLM, which is "correct" and why?
Compute the model averaged prediction on the link scale and then back-transform to the response ...
- 636
1
vote
0
answers
231
views
Model Averaging errors using lme4 in R
I am running a glmer model on a response variable with binomial distribution and random term. My data has 3 explanatory categorical variables and I have successfully run dredge() on them and their ...
- 11
4
votes
0
answers
2k
views
Model averaging in mixed models
I am reading this blog post regarding using model averaging for mixed models and the last para says
"https://sites.google.com/site/rforfishandwildlifegrads/home/mumin_usage_examples"
...
- 971
0
votes
1
answer
355
views
Model-averaged for a glm.nb [closed]
How can i do a model averaged for a negative binomial model. I try with the AICcmodavg but it is not compatible with glm.nb models.
Any help would be appreciated
Thank you very much
Magdalena
1
vote
1
answer
536
views
How does model averaging with a categorical variable work?
I have a series of models (~14) which do not include a categorical variable. One model (#15) however, does have a categorical variable with 3 levels. Normally for this model one of the categories ...
- 21
3
votes
1
answer
1k
views
McFadden Pseudo R² in averaged model
McFadden's pseudo-R² is a well-known coefficient of determination. If I am right, it can be calculated by 1-(mod_deviance/null_deviance), where ...
- 407
3
votes
1
answer
617
views
Model averaging and intercept interpretation in Splines
I have two questions regarding Multivariate Adaptive Regression splines (MARS)
1) How is intercept interpreted in MARS? In linear regression, my intercept is shown in the image below
In a linear ...
- 685
3
votes
1
answer
1k
views
Combining bagging and stacking, with and without clusters and heteroskedasticity
Question 1:
Start with the classing case of bagging, say in random forest. Fit $B$ trees to bootstrap samples of the data. Average the predictions of the $B$ trees to form a final prediction. ...
- 13.2k