# Tagged Questions

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model.

19 views

### Bayesian Variable Selection with NMIG

I have a Bayesian linear model like this: $Y_i = X_i*\beta + \epsilon_i$ . Just for completion: ($\epsilon_i \sim N(0,\sigma^2)$ $\beta \sim N_p(b_0,B_0)$, $\sigma^2 \sim Inv-Gamma (a,b)$) I would ...
18 views

### Purposeful model building for prediction and inference

What are some of the best practices and steps to building models for prediction and or inferences? What have been taught to me during my classes was the steps outlined in Chapter 4 of Hosmer et al. ...
103 views

### Is stepwise elimination of insignifcant variables invalid if applied to experimental data?

Setting: Experimental data and control-variables I want to evalute the (average) treatment effect in a randomized controlled trial. Individuals $i$ in one group received a treatment ($D_i=1$) and in ...
42 views

### covariate selection in inference problems in logistic regression

For my specific problem, but a common situation in the medical field, I have several hundred patients, and about 10-20 exaplnatory variables. the goal is to examine a specific predictor("treatment") ...
30 views

### Regression analysis method when data is linked to a normalised time… and only hold a relationship for some of it

Stats novice... Please be patient, but I would really appreciate some help. I am using SPSS. I have made some kinematic measurements of a closed chain, repeated movement. I have a large number of ...
426 views

### What are the advantages of stepwise regression?

I am experimenting with stepwise regression for the sake of diversity in my approach to the problem. So, I have 2 questions: What are the advantages of stepwise regression? What are its specific ...
42 views

### Regression result for second model still shows insignificant variables

I run a regression for my study. I used a quarterly data included 33 number of observations. my problem is independent variables are significant but not perfectly significant. when i run a ...
25 views

### Forward selection, using adjusted R square or t statistics?

When it comes to select variable in multiple regression model using forward selection, should we add variables in the models according to its adjusted R square or t statistics/Sig?
46 views

### Different variable selection techniques for Longitudinal data in R

I'm trying to perform variable selection in R and was wondering if the stepwise and Adaptive lasso codes would change for longitudinal data. Also it would be great if someone could share some sample ...
34 views

### Justification for AIC/BIC vs F-statistic when using stepwise backward elimination

I note that some stepwise backwards elimination methods use AIC to make the decision about which variables to eliminate, and others use the F-statistic. Why would I use one over the other, and is ...
55 views

### We should normalize (or standardize) data before feature selection tests (t-test, related matrix, etc.)?

We should normalize (or standardize) data before these feature selection techniques? Which one for every technique? normalization of standardization? t-test Related matrix Stepwise PCA Factor ...
54 views

### Stepwise regression in R for a quasi-poisson model

I am a little confused about the stepwise regression command in R. Is it possible that I can use it on a quasi-poisson model that I want to test out. Since the stepwise regression bases decision on ...
26 views

### What is the preferred way to present scores based on different combinations of variables?

I have a machine learning model that takes 12 variables as input. I train the model, test it on a validation set, and measure $R^2$. I have determined that a subset of the variables can be dropped ...
28 views

### Backward selection problem

In the procedure of backward selection, after dropping the variable with the largest p-value, should I do a likelihood ratio test of the reduced model with the original model first, and then check the ...
87 views

### Complete separation and stepwise regression - possible in R?

I've been using stepAIC to narrow down my logistic regression model. However, I get the following warning when I run my model: glm.fit: fitted probabilities numerically 0 or 1 occurred I know this ...
49 views

62 views

### How can I prove that the f-statistic does not follow an F distribution in the context of step-wise regression?

There is a good number of threads about the deficiencies of step-wise regression, and particularly on the shortcomings of the partial F test as a tool for step selection. However I find it difficult ...
93 views

### Interpreting AIC forward stepwise function in R

My homework asks me to: "Try a forward stepwise procedure with entry probability of 0.20. Then describe the model that is arrived at and whether it might be preferred." I used the forward step ...
59 views

### How to deal with categorical variable - location- with more than 60 levels

I am new to statistics and to categorical variables. I need to predict the cost based on several variables and it happened that all of my variables are categorical. I tried doing a linear regression ...
190 views

### Stepwise regression and variable selection with categorical variables in R

I am new with statistics and especially stepwise regression with categorical variables. I have 4 categorical variables, each with a different levels (5 levels, 12 levels, 7 levels, and 78 levels). I ...
134 views

### Recommend a method for variable selection (other than classification tree or random forest)?

Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select ...
59 views

### How to do stepwise regression forward correctly?

My understanding is that you add only one single variable at a time based on various model fit or statistical criterion. Someone advances that there is merit to running a stepwise regression by ...
121 views

### Forward selection with BIC for robust regression methods?

I want to fit a robust linear model to my data using the rlm function in R. Is there any function that provides forward model ...
887 views

### Why are p-values misleading after performing a stepwise selection?

Let's consider for example a linear regression model. I heard that, in data mining, after performing a stepwise selection based on the AIC criterion, it is misleading to look at the p-values to test ...
501 views

### Which method (enter, Forward LR or Backward LR) of logistic regression to use?

My study is a prospective observational study. My dependent variable (outcome) is development of surgical site infection (SSI) after surgery and my independent variables (predictors) are many factors ...
90 views

### Choose model by BIC in a stepwise algorithm after choosing model from glmnet

I have data where number of observation n is smaller than number of variables p. The answer variable is binary. For example: <...
240 views

### What does it mean that stepwise, backward and forward selection methods are “path dependent”?

In many papers I read that stepwise, backward and forward selection methods are "path dependent". What does it mean? Could anyone give me some practical example to understand the underlying concept? ...
77 views

### stepwise, forward and backward selection when the regressors are too much correlated

Why automated variable selection methods like stepwise regression, backward elimination and forward stepwise regression are not suitable when the regressors suffer from multicollinearity? Could anyone ...
50 views

### How does SAS's stepwise logistic work?

Not being accustomed to reading documentation as a sequence of tables, I'd like to know if someone could kindly explain how SAS's Proc Logistic invocation with selection=backwards works. http://...
356 views

### How to decide which interaction terms to include in a multiple regression model?

I am trying to build a multiple regression model using R. I have a number of predictor variables. I have some basic domain knowledge for which I am trying to build the model. To start with, I included ...
71 views

### R stepAIC multiple datasets

I have three datasets of similar(ish) sizes (268, 271 and 262), and a model I am trying to develop for describing a response variable within the data. I'm trying to use the ...
89 views

### Using correlation to eliminate predictors? [duplicate]

I have 1 dependent variable and 33 independent variables (continuous, categorical & dichotomous). Correlation analyses (2-tailed) show that the DV is only correlated to 7 of the IVs although most ...
40 views

### Can a variable become statistically significant after the addition of another variable? [duplicate]

I am doing forward stepwise logistic regression. I have heard that its common for a previously statistically significant variable to become not statistically significant when one or more variables are ...
29 views

### Is it possible to use stepwise linear regression to find the parameters that explain the most variation in the output of a nonlinear system?

I have a mathematical model of a system. We have to use a simulation software to check the response of the system to specific input signals or change of model parameters. The relationship between some ...
127 views

### Controlling for age and sex in a multiple regression with a backward model selection

So, I have this dataset with a hormonal measure as independent variable and behavioral measures as dependent variables. I am using a linear regression as my model and after a backward selection, ended ...
11 views

### Detecting outlying distributions of ratio data

I have a dataset consisting of hundreds of repeat observations on thousands of agents. Each observation is a ratio between two distance measures, A and B, where A is always larger than B. Thus, my ...
666 views

### Combining principal component regression and stepwise regression

I want to use a combination of principal component analysis (PCA) and stepwise regression to develop a predictor model. I have 5 independent variables (which are correlated among each other to ...
142 views

### R: Dynamic Regression with ARIMA model using xreg, make use of step function?

This might fit better here than on stackoverflow, I guess. I was trying to build a dynamic regression model with the dynlm package, but it did not work out. After reading this by Hyndman, I now ...
302 views

### Significant predictors of mpg in mtcars dataset in R

Which variables are significant predictors of mpg in mtcars dataset. When I perform following regression: ...
581 views

### What is the difference between VIF and stepwise regression?

What is the difference between the variance inflation factor (VIF) and stepwise regression as both help in detecting multicollinearity? What variables are different while running both techniques?
92 views

### Do I drop insignificant parameters from a model? Should I use stepwise regression?

I'm working on a project where we have a number of factors we believe might have a role on a survey result. It's my job to figure out if this is true. My boss suggested just doing correlations on each ...
185 views

### R: Why does step function of a Linear Modegives different AIC/BIC than AIC function?

I don't understand what I make or think wrong, but if one tries to evaluate the linear model of the data (which you can find in R in the Package AIC(stats)), then ...
398 views

### How to use residual analysis to remove the effect of confounding variables in a model in R

I want to find which soil variables better explain plant productivity, using a database that contains information for about 100 forests plots across Europe. These plots have only one species per plot, ...