Questions tagged [model-averaging]

The process of combining different models to get a better resulting model than any of the constituents. Eg, computing a parameter estimator as the average of the estimators from each component model.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

No random effect variance estimated from model averaging?

When averaging multiple GLMM models which include random effect no estimate of random effect variance is given in the results of the averaged model, why is this? I assume there is a good reason, but I ...
0
votes
0answers
26 views

Bayesian model averaging when none of the models is well specified

Usually, from what I've read of Bayesian Model Averaging (BMA), a typical assumption is that one of the models is well specified... However, what will happen when all models are misspecified? What ...
2
votes
0answers
36 views

Why is pseudo-Bayesian Model Averaging using WAIC giving counterintuitive results for my model? Issue with unconstrained data uncertainty?

I'm currently working on a model which fits a pair of curves which are parameterized by some shape and scaling parameters. I want to produce weights for two different models. In my first model, which ...
4
votes
1answer
87 views

Regularization via model averaging?

Say you have the model $$ \Phi^{-1} \left(y\right)=\beta_0 + \beta_1 x$$ I am interested in adding some regularization, specifically concerning the parameter $\beta_1$, to introduce some "skepticism" ...
1
vote
0answers
51 views

Is It Valid To Calculate Model-Averaged Confidence Intervals In the Same Way As Model Averaged Predictions?

I'm fitting a series of mixed-effects models, and I'm trying to calculate the model-averaged predictions and their confidence intervals. If I have a set of $R$ models $\{M_1,...,M_R\}$ I know that ...
0
votes
0answers
36 views

When averaging models, which models need to meet assumptions?

I have 16 variables and am running all possible models (65,535 total!), then averaging the best models. A model including all variables has normally distributed residuals, but some of the 65,535 ...
1
vote
1answer
108 views

Is there a difference between averaging individual regressions and including a random effect?

I have a bit of a theoretical question about random effects models and regression. If I have a set of clustered, longitudinal data (say repeated measurements of $y$ on a number of different ...
0
votes
0answers
19 views

Coefficient Averaging to Predict Value at end of Time-Series/Possible Random Effects Model?

I have some discrete time series data that consist of the following variables of interest: Project ID ($project\_id$) The total budget for a particular project ($tot\_budget$) The person who runs the ...
1
vote
0answers
47 views

Combining regression models from separate data sets

What is the best way to combine regression betas from separate data sets? For example, a data set is split in two based on some fundamental characteristic, and the same two factor regression is run ...
2
votes
2answers
214 views

Weighting of multiple linear regressions in an ensemble

If I have a continuous dependent variable and N continuous predictors, and I fit all possible regressions with zero up to N variables, how should I weight those regressions for prediction? One ...
1
vote
0answers
57 views

Is this Bayesian model averaging?

A classical example of Bayesian model averaging (BMA) is the regression setup where the choice of different sets of covariates corresponds to different models $\mathcal{M}_k$, $k = 1, \ldots, K$, ...
1
vote
0answers
114 views

Can model averaging be applied on models fitted to different data sets?

I have two data sets collected from two different sets of participants on their behaviour. EDIT: Both have the same response variables (Propensity to behave in a certain way - Yes or No). But they ...
1
vote
1answer
67 views

BMA - at what posterior effect probability can we say that a variable has an effect?

in the setting of a Bayesian model averaging, we are not dealing with P--value when we are assesing variable importance, but with the posterior effect probability of each variable. My question is, at ...
0
votes
1answer
72 views

Interpretation of a classic multinomial logit vs. BMA of multinomial logit

EDIT #1 Most likely I have set up the function bic.mlogit in a wrong way. @Jesper Hybel, hopefully, directed me in the right way. With the new setup I get two sets ...
0
votes
1answer
55 views

vector of weights

I am using the MuMIn package for model averaging. However, I am not clear of the function par.avg(). In this function, we need to specify the following, ...
2
votes
1answer
431 views

Effect sizes for model averaging

The goal is to find if one factor is stronger than the other in the models I have considered. I am using the information-theoretic approach. Since $n/K>40$, I am using AIC. Firstly the model is ...
12
votes
1answer
638 views

For model-averaging a GLM, do we average the predictions on the link or response scale?

To compute the model-averaged predictions on the response scale of a GLM, which is "correct" and why? Compute the model averaged prediction on the link scale and then back-transform to the response ...
1
vote
0answers
80 views

Model Averaging errors using lme4 in R

I am running a glmer model on a response variable with binomial distribution and random term. My data has 3 explanatory categorical variables and I have successfully run dredge() on them and their ...
1
vote
0answers
915 views

Model averaging in mixed models

I am reading this blog post regarding using model averaging for mixed models and the last para says "https://sites.google.com/site/rforfishandwildlifegrads/home/mumin_usage_examples" ...
0
votes
1answer
211 views

Model-averaged for a glm.nb [closed]

How can i do a model averaged for a negative binomial model. I try with the AICcmodavg but it is not compatible with glm.nb models. Any help would be appreciated Thank you very much Magdalena
0
votes
1answer
275 views

How does model averaging with a categorical variable work?

I have a series of models (~14) which do not include a categorical variable. One model (#15) however, does have a categorical variable with 3 levels. Normally for this model one of the categories ...
2
votes
1answer
470 views

McFadden Pseudo R² in averaged model

McFadden's pseudo-R² is a well-known coefficient of determination. If I am right, it can be calculated by 1-(mod_deviance/null_deviance), where ...
2
votes
1answer
291 views

Model averaging and intercept interpretation in Splines

I have two questions regarding Multivariate Adaptive Regression splines (MARS) 1) How is intercept interpreted in MARS? In linear regression, my intercept is shown in the image below In a linear ...
2
votes
1answer
763 views

Combining bagging and stacking, with and without clusters and heteroskedasticity

Question 1: Start with the classing case of bagging, say in random forest. Fit $B$ trees to bootstrap samples of the data. Average the predictions of the $B$ trees to form a final prediction. ...
0
votes
0answers
22 views

Can you increase confidence from two similar predictions based on independent data

Let's say I have two models to predict the out come of a game. Model A uses the player specific statistics (speed, reaction times etc.) and predicts a 75% chance of a win for my team. Model B uses ...
1
vote
0answers
325 views

Confidence interval of averaged-regression fit

I have a model; Y ~ mX, where Y is a set of new random number generated using Poisson distribution. i.e $Y_i \sim \text{poisson}(H_i)$. Hence I am performing 100 replications, during each replication,...
0
votes
1answer
26 views

Is the average of multiple function's minimum equivalent to the minimum of the average function of the multiple functions?

Sorry if the wording is confusing, I feel it accurately articulates the question although being a bit of a head teaser. Essentially I am running many cross-validated models. Each model produces a (...
2
votes
3answers
221 views

Models combinations

My goal is time series forecasting. I have created a number of models to make predictions. I know that forecast quality can be improved by combining predictions from different models(linear ...
2
votes
0answers
1k views

One multiple linear regression, or many simple linear regressions for prediction?

I would like to create a model which predicts the amount of energy used in an area, dependent on the number of properties in 5 categories (detached, semi, flats, bungalow and terrace). I have daily ...
2
votes
1answer
62 views

Variable Selection and Model Likelihood

Can I derive any (Bayesian) statements about the probability of a parameter given the data and the marginal likelihood of models containing this parameter? For instance, consider a regression setup $$...
2
votes
0answers
100 views

Ensemble, what are simple alternatives to averaging?

I have a classification problem (A or B or C). I am currently evaluating test set results from the trained random forest, neural net, and logistic regression models. Any one model works pretty well ...
1
vote
0answers
116 views

Panal data, simple model averaging in R

For my master thesis I working on model averaging for panel data. So far I was not able to use common model averaging packages (BMS, BMA) because I use panel data (plm regression first differences). ...
3
votes
3answers
103 views

Arguments against model or forecast combination?

Do you know any references providing arguments against model or forecast (models output) combination? Could not find anything
2
votes
1answer
435 views

Do model averaging and model combination mean the same?

I am not sure, but I guess model averaging and model combination and even forecast averaging and forecast combination are used arbitrarily in the literature... Is this only my feeling or indeed the ...
6
votes
1answer
2k views

Use BIC or AIC as approximation for Bayesian Model Averaging [duplicate]

I want to compare "real" Bayesian Model Averaging (BMA) performed with the EM algorithm and information-criterion based BMA. Which one, BIC or AIC, is a "closer" approximation to the "real" BMA? BIC ...
5
votes
1answer
837 views

Are these approaches Bayesian, Frequentist or both?

currently I am comparing different combination methods: Equally Weight Averaging -> deterministic Ordinary Least Squares Averaging -> frequentist? Bayesian Information Criterion Averaging -> both, ...
1
vote
1answer
75 views

Bayesian model averaging for beta estimation

I am trying to apply BMA to estimate the beta of a certain stock by combining different models. One, simple model to estimate beta is this $$\beta_{i,T} = \frac{{\rm cov}(r_{i,t},r_{M,t})}{{\rm var}(...
2
votes
2answers
425 views

Bayesian Model Averaging: How to use in this example?

Let ${M_1, M_2}$ denote two competing forecasting models. With Bayesian model averaging we can get $p(y_{T+h}|y_{1:T}) = \sum_{j=1}^2p(y_{T+h}|y_{1:T},M_j)*p(M_j|y_{1:T})$ $1:T$ represents the ...
9
votes
1answer
1k views

Model averaging approach — averaging coefficient estimates vs. model predictions?

I have a basic question regarding approaches to model averaging using IT criteria to weight models within a candidate set. Most sources that I have read on model averaging advocate averaging the ...
4
votes
1answer
317 views

Bayesian Model Averaging with MCMC draws

I aim at improving my understanding of Bayesian model averaging in the context of predictions and I am somewhat stuck on how to implement this numerically. Lets say I have two models $\mathcal{M}_1$ ...
6
votes
1answer
589 views

Weights to combine different models

I have built different classification models (logistic regression, randomforest, and xgboost) for a dataset. I would like to combine the prediction of all the models to reduce the variance and ...
3
votes
1answer
242 views

What exactly is the Bayesian model average probability of the following?

I am struggling to understand Bayesian Model Averaging. Suppose I have got a simple Bayesian network, $X\to Y$ where X and Y are binary variables and I have got 2 models: Model 1 (simple): $$P(Y=1\...
0
votes
2answers
98 views

Model averaging

We have the equation: $P(C|X)=\sum_{i}P(C|X,M_{i})P(M_{i}|X)$ I should technically be able to prove that it works even when the total number of models is 1, so I go from the right hand side of the ...
0
votes
1answer
389 views

Calculating QAICc and Averaging GLMM models with various overdispersion

I am having difficulty figuring out how to calculate a dispersion parameter to calculate QAICc for a GLMM with a binomial fit. I have tested for overdispersion using this code: ...
3
votes
0answers
959 views

How to implement Bayesian Model Combination?

I'm interested in formal procedure mentioned in "Turning Bayesian Model Averaging Into Bayesian Model Combination" (Kristine Monteith 2011). I have a set of $N$ "best" AIC ranked models and I want to ...
12
votes
2answers
11k views

Interpreting model averaging results in R

I am trying to understand and know what to report from my analysis of some data using model averaging in R. I am using the following script to analyse the effect of method of measurement over a given ...
1
vote
0answers
524 views

Model Averaging: Standard Error vs Adjusted Standard Error

I'm having trouble understanding when to use Std. Error or Adjusted SE in model averaging (model.avg). What/Why/When to use Adjusted SE? This is my code: ...
2
votes
1answer
1k views

Confidence bands for model averaged predictions of GLMMs

I use R with the MuMIn package for Multimodel inference. My global Model is ...
2
votes
0answers
75 views

What would be a proper statistical test for multiple models and model selection?

Suppose I have a data set of N observations (n = 1...N) for out-of-sample estimation and values of ($y_n$). I have also ...
2
votes
1answer
307 views

Is it inherently invalid to use BIC for model averaging?

If I understand AIC/AICc vs. BIC correctly, AIC presumes that there is no "true" model and that any given model is simply a "best worst approximation". However, BIC presumes that there IS a "true" ...