Questions tagged [model-averaging]

The process of combining different models to get a better resulting model than any of the constituents. Eg, computing a parameter estimator as the average of the estimators from each component model.

Filter by
Sorted by
Tagged with
4 votes
1 answer
47 views

Bayesian model averaging

In which situation would you refrain from using BMA. It seems to me it is always a good idea to use the posterior probabilities when infering/predicting.
user avatar
  • 43
0 votes
1 answer
54 views

When/what to standardize for model-averaging with(out) interactions

I’m using the {MuMIn} package in R to select models (dredge, get top models, average etc). My question is about whether I need to, or should, standardise my ...
user avatar
  • 1
10 votes
1 answer
825 views

Averaging SVM and GLM results: sensible or stupid?

I have taken two different approaches to calculate probability: using a GLM and an SVM. They are giving slightly different results (which is understandable, they are completely different approaches). ...
user avatar
1 vote
0 answers
22 views

BMA and RJMCMC predictive performance

We have a family of statistical models, with parameter spaces of different dimensions, which we aggregated through standard Bayesian Model Averaging (BMA). Experiments using training and test sets ...
user avatar
  • 22.2k
2 votes
1 answer
61 views

Does bagging work for OLS to improve prediction?

In Elements of Statistical Learning, section 8.7, the author states that The bagged estimate will differ from original estimate only when the latter is a nonlinear or adaptive function of the data I ...
user avatar
  • 151
10 votes
4 answers
1k views

In what applications do we prefer Model Selection over Model Averaging?

I'm wondering in what applications or scenarios (or in trying to answer what kind of questions), the researcher would prefer using Model Selection (such as AIC or BIC) over Model Averaging (such as ...
user avatar
  • 1,734
4 votes
1 answer
87 views

Is post-variable-selection multimodel inference a bad idea?

If I understood correctly, in this answer, Ben Bolker says that using inferential methods after having performed AIC-based model selection is wrong because "standard inferential methods assume ...
user avatar
  • 439
0 votes
0 answers
14 views

Studentized residuals with model averaging

With a standard regression model, we might obtain externally Studentized residuals to identify the worst outliers. But suppose in addition we have lots of possible predictors, all inter-correlated, so ...
user avatar
  • 16.5k
0 votes
0 answers
15 views

How can model averaging be done without or only a few known examples?

Model averaging has been widely used to combine multiple predictions from single models that are prone to statistical bias. As far as I know, the commonly used averaging models, including Bates-...
user avatar
5 votes
2 answers
216 views

Do "conditional" averaged coefficients *ever* make sense?

This question is related to one on Stack Overflow where the user wanted to obtain predictions and their standard errors using the "conditional" averaged coefficients from the R package MuMIn....
user avatar
  • 16.5k
1 vote
0 answers
97 views

Is model averaging possible/appropriate with a GAM?

I'm new here - apologies if there are any problems. I have two GAMs that I've fit (using mgcv::gam()), and the two models are comparable in AICc values. With GLMs, ...
user avatar
  • 11
0 votes
0 answers
32 views

Averaging partial dependence plots of multiple BRT models

I have built multiple BRT models (using the dismo and gbm packages in R) using different random subsets of a dataset. I would like to create partial dependence plots for each variable using the mean +/...
user avatar
  • 1
1 vote
0 answers
165 views

How does the R function confint calculate confidence intervals for a model averaged object from MuMIn?

I've been using the MuMin package in R to create model averaged estimates from a candidate model set and the documentation suggests that you can use confint() to create confidence intervals for the ...
user avatar
  • 11
2 votes
1 answer
141 views

Can Bayesian Model Averaging be Optimal when the Hypothesis Space does not contain the true hypothesis?

I am utterly confused. I have been reading about the optimality of Bayes classifier and Bayesian model averaging all the time, but when I try to dig deeper, I just get more confused. On the one hand, ...
user avatar
1 vote
3 answers
92 views

How can we assume the models are exhaustive in Bayesian Model Averaging?

Bayesian model averaging is justified using the law of total probability which requires the the set of models that we average over to be exhaustive. Shouldn’t we prove that the set of models are ...
user avatar
1 vote
0 answers
56 views

Model averaging parameter estimates for glmer models using AICmodavg in R

I encounter the following error trying to calculate model-averaged parameter estimates across a set of GLMM models using the AICmodavg package version 2.3.1. in RStudio(version1.2.5) ERROR: Error in ...
user avatar
1 vote
0 answers
146 views

When is it acceptable to compute (conditional) subset-averaged coefficients?

I'm running an ecological study and I have 4 dependent variables (DVs) that I would like to explain (my interest thus lies in inference and not in prediction). For each one of these variables, I built ...
user avatar
  • 439
0 votes
0 answers
79 views

Model averaging a GLM function with equal weights

Background Let me first say that I read this post and that I looked at the BMS vignette. I used the package sure (CRAN, R ...
user avatar
  • 259
1 vote
0 answers
56 views

Bayesian Model Averaging (Bayesian Averaging of Classical Estimates) Issue

I am trying to implement a method used by this paper (Described briefly at the bottom of page 3): From what I understand, it does 2$^k$ OLS regressions each time step to forecast the next time step ...
user avatar
  • 11
2 votes
1 answer
50 views

Is it a good idea to use a linear model (like logistic regression) to generate new features for a non linear model (like random forest)? [duplicate]

The setting is a 2-class classification problem. We have too many features, some of them not very informative and with many zeros. We are thinking in ways of selecting the best features, and PCA (in ...
user avatar
8 votes
1 answer
276 views

AIC model averaging when models are correlated

AIC model-averaging: In "standard" AIC model averaging we average models with weights proportional to $$w_i \propto \exp( -0.5 \times \Delta \text{AIC}_i ),$$ where $\Delta \text{AIC}_i$ is ...
user avatar
  • 23.5k
1 vote
0 answers
37 views

Using AIC weights to determine prediction intervals for a single model structure

I am working with fitting regression models to data, and producing prediction intervals would be useful. Unfortunately, the data often has few data points, and is reported as mean rather than ...
user avatar
  • 41
3 votes
1 answer
1k views

Model averaging: predictor significance vs importance

What is the correct way to report the results of model averaging? I’m using MuMin in R, results include a p-value. It’s also possible to get the importance of each predictor. Example results below. ...
user avatar
  • 115
0 votes
0 answers
41 views

Bayesian model averaging when none of the models is well specified

Usually, from what I've read of Bayesian Model Averaging (BMA), a typical assumption is that one of the models is well specified... However, what will happen when all models are misspecified? What ...
user avatar
2 votes
0 answers
127 views

Why is pseudo-Bayesian Model Averaging using WAIC giving counterintuitive results for my model? Issue with unconstrained data uncertainty?

I'm currently working on a model which fits a pair of curves which are parameterized by some shape and scaling parameters. I want to produce weights for two different models. In my first model, which ...
user avatar
  • 21
4 votes
1 answer
184 views

Regularization via model averaging?

Say you have the model $$ \Phi^{-1} \left(y\right)=\beta_0 + \beta_1 x$$ I am interested in adding some regularization, specifically concerning the parameter $\beta_1$, to introduce some "skepticism" ...
user avatar
  • 2,791
1 vote
0 answers
259 views

Is It Valid To Calculate Model-Averaged Confidence Intervals In the Same Way As Model Averaged Predictions?

I'm fitting a series of mixed-effects models, and I'm trying to calculate the model-averaged predictions and their confidence intervals. If I have a set of $R$ models $\{M_1,...,M_R\}$ I know that ...
user avatar
0 votes
0 answers
111 views

When averaging models, which models need to meet assumptions?

I have 16 variables and am running all possible models (65,535 total!), then averaging the best models. A model including all variables has normally distributed residuals, but some of the 65,535 ...
user avatar
1 vote
1 answer
431 views

Is there a difference between averaging individual regressions and including a random effect?

I have a bit of a theoretical question about random effects models and regression. If I have a set of clustered, longitudinal data (say repeated measurements of $y$ on a number of different ...
user avatar
1 vote
0 answers
122 views

Combining regression models from separate data sets

What is the best way to combine regression betas from separate data sets? For example, a data set is split in two based on some fundamental characteristic, and the same two factor regression is run ...
user avatar
2 votes
2 answers
832 views

Weighting of multiple linear regressions in an ensemble

If I have a continuous dependent variable and N continuous predictors, and I fit all possible regressions with zero up to N variables, how should I weight those regressions for prediction? One ...
user avatar
1 vote
0 answers
85 views

Is this Bayesian model averaging?

A classical example of Bayesian model averaging (BMA) is the regression setup where the choice of different sets of covariates corresponds to different models $\mathcal{M}_k$, $k = 1, \ldots, K$, ...
user avatar
  • 405
1 vote
0 answers
208 views

Can model averaging be applied on models fitted to different data sets?

I have two data sets collected from two different sets of participants on their behaviour. EDIT: Both have the same response variables (Propensity to behave in a certain way - Yes or No). But they ...
user avatar
  • 11
1 vote
1 answer
130 views

BMA - at what posterior effect probability can we say that a variable has an effect?

in the setting of a Bayesian model averaging, we are not dealing with P--value when we are assesing variable importance, but with the posterior effect probability of each variable. My question is, at ...
user avatar
  • 105
0 votes
1 answer
140 views

Interpretation of a classic multinomial logit vs. BMA of multinomial logit

EDIT #1 Most likely I have set up the function bic.mlogit in a wrong way. @Jesper Hybel, hopefully, directed me in the right way. With the new setup I get two sets ...
user avatar
  • 105
0 votes
1 answer
80 views

vector of weights

I am using the MuMIn package for model averaging. However, I am not clear of the function par.avg(). In this function, we need to specify the following, ...
user avatar
  • 71
2 votes
1 answer
1k views

Effect sizes for model averaging

The goal is to find if one factor is stronger than the other in the models I have considered. I am using the information-theoretic approach. Since $n/K>40$, I am using AIC. Firstly the model is ...
user avatar
  • 71
13 votes
1 answer
1k views

For model-averaging a GLM, do we average the predictions on the link or response scale?

To compute the model-averaged predictions on the response scale of a GLM, which is "correct" and why? Compute the model averaged prediction on the link scale and then back-transform to the response ...
user avatar
  • 559
1 vote
0 answers
190 views

Model Averaging errors using lme4 in R

I am running a glmer model on a response variable with binomial distribution and random term. My data has 3 explanatory categorical variables and I have successfully run dredge() on them and their ...
user avatar
  • 11
4 votes
0 answers
2k views

Model averaging in mixed models

I am reading this blog post regarding using model averaging for mixed models and the last para says "https://sites.google.com/site/rforfishandwildlifegrads/home/mumin_usage_examples" ...
user avatar
  • 911
0 votes
1 answer
274 views

Model-averaged for a glm.nb [closed]

How can i do a model averaged for a negative binomial model. I try with the AICcmodavg but it is not compatible with glm.nb models. Any help would be appreciated Thank you very much Magdalena
user avatar
1 vote
1 answer
457 views

How does model averaging with a categorical variable work?

I have a series of models (~14) which do not include a categorical variable. One model (#15) however, does have a categorical variable with 3 levels. Normally for this model one of the categories ...
user avatar
3 votes
1 answer
961 views

McFadden Pseudo R² in averaged model

McFadden's pseudo-R² is a well-known coefficient of determination. If I am right, it can be calculated by 1-(mod_deviance/null_deviance), where ...
user avatar
  • 406
3 votes
1 answer
516 views

Model averaging and intercept interpretation in Splines

I have two questions regarding Multivariate Adaptive Regression splines (MARS) 1) How is intercept interpreted in MARS? In linear regression, my intercept is shown in the image below In a linear ...
user avatar
  • 665
3 votes
1 answer
1k views

Combining bagging and stacking, with and without clusters and heteroskedasticity

Question 1: Start with the classing case of bagging, say in random forest. Fit $B$ trees to bootstrap samples of the data. Average the predictions of the $B$ trees to form a final prediction. ...
user avatar
  • 12.3k
0 votes
0 answers
48 views

Can you increase confidence from two similar predictions based on independent data

Let's say I have two models to predict the out come of a game. Model A uses the player specific statistics (speed, reaction times etc.) and predicts a 75% chance of a win for my team. Model B uses ...
user avatar
1 vote
0 answers
345 views

Confidence interval of averaged-regression fit

I have a model; Y ~ mX, where Y is a set of new random number generated using Poisson distribution. i.e $Y_i \sim \text{poisson}(H_i)$. Hence I am performing 100 replications, during each replication,...
user avatar
0 votes
1 answer
45 views

Is the average of multiple function's minimum equivalent to the minimum of the average function of the multiple functions?

Sorry if the wording is confusing, I feel it accurately articulates the question although being a bit of a head teaser. Essentially I am running many cross-validated models. Each model produces a (...
user avatar
2 votes
3 answers
293 views

Models combinations

My goal is time series forecasting. I have created a number of models to make predictions. I know that forecast quality can be improved by combining predictions from different models(linear ...
user avatar
3 votes
0 answers
2k views

One multiple linear regression, or many simple linear regressions for prediction?

I would like to create a model which predicts the amount of energy used in an area, dependent on the number of properties in 5 categories (detached, semi, flats, bungalow and terrace). I have daily ...
user avatar
  • 437