# Tag Info

5

BIC and AIC can both be regarded as (asymptotically) Bayesian (under different situations), but they can also be regarded just as penalized (/regularized) maximum likelihood, for example. That is, using BIC to do model selection doesn't mean you're being inherently Bayesian; a frequentist might choose to use it based on the fact that in many situations its ...

3

(1) There is an extensive literature on why one should prefer full models to restricted/parsimonious models. My understanding are few reasons to prefer the parsimonious model. However, larger models may not be feasible for many clinical applications. (2) As far as I know, Discrimination/Discrimination indexes aren’t (?should not be) used as a ...

2

One option is to use pseudo R-square measures for both models. A strong difference in pseudo R-square would suggest that the model fit strongly decreases by omitting V17. There are different kinds of Pseudo R-squares available. An overview can be found here, for example: http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm A popular ...

2

I have also another take, based on the book by Anderson on model selection. The authors state as follows: It seems best not to associate the words significant or rejected with results under an information-theoretic paradigm. Questions concerning the strength of evidence for the models in the set are best addressed using the evidence ratio as well ...

2

Just some strayed thoughts. 1) You fit a multiple regression to examine the effect of a particular variable a worker in another department is interested in. The variable comes back insignificant, but your co-worker says that this is impossible as it is known to have an effect. What would you do? Many reasons could have caused this: The study is ...

2

Probably what you will need to use is the Parametric Bootstrap Cross-fitting Method. Here is the basic procedure: Fit each model to the data. Estimate the models' parameters and extract your favorite measure of goodness of fit. We will call the model with the higher value for this GoF measure $A$ and the other model $B$. Calculate the difference $d$ ...

2

You can think of AIC as a providing a more reasonable (i.e., larger) $P$-value cutoff. But model selection based on $P$-values or any other one-variable-at-a-time metric is frought with difficulties, having all the problems of stepwise variable selection. Generally speaking, AIC works best if used to select a unique single parameter (e.g., shrinkage ...

2

I would say it is often appropriate to use AIC in model selection, but rarely right to use it as the sole basis for model selection. We must also use substantive knowledge. In your particular case, you are comparing a model with a 3rd order AR vs. one with a 1st order AR. In addition to AIC (or something similar) I would look at the the autocorrelation and ...

2

(1) Do you really need a smaller model? If not, you're set. (2) Can you honestly pre-specify your model? From your knowledge of the field can you choose a subset of predictors your interested in without using your knowledge of this dataset? If so, you're set. (2.5) If all your data valid? Assess this without looking at outcomes. (3) Consider using some form ...

1

Do not perform backward selection. Nor forward. Nor stepwise. Don't do it for Cox or OLS or logistic or any other model. Instead, think about what variables you have, why you have them, what they mean, what theory says about them, how including them affects other variables and so on. If you must use some automated method, LASSO or LAR have nice properties. ...

1

The flat maximum effect is simply that models with very different coefficients can have very similar performance on a particular prediction or classification task. When this happens it makes interpreting model coefficients difficult. One example of where this occurs is in linear regression with multicolinearity of the predictor variables. As another ...

1

You would do better to avoid such a thing because it (very rarely) makes sense. What would make sense in your situation? Well, since you have only three continuous IVs and one categorical one, unless your N is very small indeed, there is little problem with keeping all the variables. Also, you wrote you want to investigate whether there is "any ...

1

1) This is a multiple regression. Presumably there are other variables. It may be the case that the specific variable in question has an effect on the response, but in the presence of the other variables its effect is not significant. I'd use partial least squares pairing the variable in question with the other variables to determine which other variables ...

1

(1) This doesn't quite make sense. "What would you do" is very general. Often variables that are known to be significant/related to the outcome are included in a regression model even if they are found not to be significant (to increase confidence in that you are providing unbiased estimates for the other coefficients). So one answer would be you keep the ...

Only top voted, non community-wiki answers of a minimum length are eligible