Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model.

learn more… | top users | synonyms

1
vote
0answers
27 views

Regression analysis method when data is linked to a normalised time… and only hold a relationship for some of it

Stats novice... Please be patient, but I would really appreciate some help. I am using SPSS. I have made some kinematic measurements of a closed chain, repeated movement. I have a large number of ...
6
votes
3answers
402 views

What are the advantages of stepwise regression?

I am experimenting with stepwise regression for the sake of diversity in my approach to the problem. So, I have 2 questions: What are the advantages of stepwise regression? What are its specific ...
0
votes
1answer
42 views

Regression result for second model still shows insignificant variables

I run a regression for my study. I used a quarterly data included 33 number of observations. my problem is independent variables are significant but not perfectly significant. when i run a ...
0
votes
0answers
22 views

Forward selection, using adjusted R square or t statistics?

When it comes to select variable in multiple regression model using forward selection, should we add variables in the models according to its adjusted R square or t statistics/Sig?
0
votes
0answers
38 views

Different variable selection techniques for Longitudinal data in R

I'm trying to perform variable selection in R and was wondering if the stepwise and Adaptive lasso codes would change for longitudinal data. Also it would be great if someone could share some sample ...
1
vote
0answers
32 views

Justification for AIC/BIC vs F-statistic when using stepwise backward elimination

I note that some stepwise backwards elimination methods use AIC to make the decision about which variables to eliminate, and others use the F-statistic. Why would I use one over the other, and is ...
0
votes
0answers
44 views

We should normalize (or standardize) data before feature selection tests (t-test, related matrix, etc.)?

We should normalize (or standardize) data before these feature selection techniques? Which one for every technique? normalization of standardization? t-test Related matrix Stepwise PCA Factor ...
0
votes
0answers
42 views

Stepwise regression in R for a quasi-poisson model

I am a little confused about the stepwise regression command in R. Is it possible that I can use it on a quasi-poisson model that I want to test out. Since the stepwise regression bases decision on ...
1
vote
0answers
24 views

What is the preferred way to present scores based on different combinations of variables?

I have a machine learning model that takes 12 variables as input. I train the model, test it on a validation set, and measure $R^2$. I have determined that a subset of the variables can be dropped ...
0
votes
0answers
28 views

Backward selection problem

In the procedure of backward selection, after dropping the variable with the largest p-value, should I do a likelihood ratio test of the reduced model with the original model first, and then check the ...
0
votes
1answer
85 views

Complete separation and stepwise regression - possible in R?

I've been using stepAIC to narrow down my logistic regression model. However, I get the following warning when I run my model: glm.fit: fitted probabilities numerically 0 or 1 occurred I know this ...
1
vote
0answers
49 views

Convergence analysis for forward stagewise regression?

Forward stagewise regression is a simple model selection algorithm related to least angle regression and LASSO. (see e.g. the LARS paper) It repeats the following steps, initializing a predictor $\hat{...
0
votes
0answers
41 views

Random Forests with almost 200 predictors

I have a data set I'm playing around with that has almost 200 independent variables. What is the best way to limit this down? My response in binomial so I was thinking making I could use a GLM and ...
0
votes
0answers
44 views

How is cross-validation used for logistic regression?

I have a fundamental question about cross-validation in logistic regression. I would really appreciate some help since something is still unclear to me. My situation is the following: I split my data ...
1
vote
1answer
71 views

Is it valid to get better performance in logistic regression using only a subset of the coefficients?

I have an imbalanced data set containing 12% of the positive class 88% negative. First, I ran a logistic regression with all my coefficients and got an average accuracy of 0.91 (I know that's not ...
0
votes
1answer
127 views

Is multicollinearity an issue when doing stepwise logistic regression using AIC and BIC?

As far as I understood, it should not be a problem as long as I don't have perfect multicollinearity since I don't mind if the standard errors get inflated. However, what about using the Likelihood-...
4
votes
1answer
141 views

Linear regression - iterative approach

I have a single output variable $y$ and a number of inputs $x_1$, $x_2$, etc. These are time series. Each $x_i$ explains the changes in $y$ in specific circumstances, and the goal is to have a linear ...
3
votes
1answer
58 views

How can I prove that the f-statistic does not follow an F distribution in the context of step-wise regression?

There is a good number of threads about the deficiencies of step-wise regression, and particularly on the shortcomings of the partial F test as a tool for step selection. However I find it difficult ...
0
votes
0answers
84 views

Interpreting AIC forward stepwise function in R

My homework asks me to: "Try a forward stepwise procedure with entry probability of 0.20. Then describe the model that is arrived at and whether it might be preferred." I used the forward step ...
0
votes
1answer
58 views

How to deal with categorical variable - location- with more than 60 levels

I am new to statistics and to categorical variables. I need to predict the cost based on several variables and it happened that all of my variables are categorical. I tried doing a linear regression ...
0
votes
0answers
158 views

Stepwise regression and variable selection with categorical variables in R

I am new with statistics and especially stepwise regression with categorical variables. I have 4 categorical variables, each with a different levels (5 levels, 12 levels, 7 levels, and 78 levels). I ...
2
votes
1answer
131 views

Recommend a method for variable selection (other than classification tree or random forest)?

Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select ...
0
votes
0answers
57 views

How to do stepwise regression forward correctly?

My understanding is that you add only one single variable at a time based on various model fit or statistical criterion. Someone advances that there is merit to running a stepwise regression by ...
0
votes
1answer
106 views

Forward selection with BIC for robust regression methods?

I want to fit a robust linear model to my data using the rlm function in R. Is there any function that provides forward model ...
15
votes
2answers
879 views

Why are p-values misleading after performing a stepwise selection?

Let's consider for example a linear regression model. I heard that, in data mining, after performing a stepwise selection based on the AIC criterion, it is misleading to look at the p-values to test ...
0
votes
1answer
471 views

Which method (enter, Forward LR or Backward LR) of logistic regression to use?

My study is a prospective observational study. My dependent variable (outcome) is development of surgical site infection (SSI) after surgery and my independent variables (predictors) are many factors ...
0
votes
0answers
81 views

Choose model by BIC in a stepwise algorithm after choosing model from glmnet

I have data where number of observation n is smaller than number of variables p. The answer variable is binary. For example: <...
3
votes
1answer
226 views

What does it mean that stepwise, backward and forward selection methods are “path dependent”?

In many papers I read that stepwise, backward and forward selection methods are "path dependent". What does it mean? Could anyone give me some practical example to understand the underlying concept? ...
0
votes
1answer
69 views

stepwise, forward and backward selection when the regressors are too much correlated

Why automated variable selection methods like stepwise regression, backward elimination and forward stepwise regression are not suitable when the regressors suffer from multicollinearity? Could anyone ...
1
vote
0answers
49 views

How does SAS's stepwise logistic work?

Not being accustomed to reading documentation as a sequence of tables, I'd like to know if someone could kindly explain how SAS's Proc Logistic invocation with selection=backwards works. http://...
1
vote
1answer
330 views

How to decide which interaction terms to include in a multiple regression model?

I am trying to build a multiple regression model using R. I have a number of predictor variables. I have some basic domain knowledge for which I am trying to build the model. To start with, I included ...
0
votes
0answers
68 views

R stepAIC multiple datasets

I have three datasets of similar(ish) sizes (268, 271 and 262), and a model I am trying to develop for describing a response variable within the data. I'm trying to use the ...
1
vote
2answers
84 views

Using correlation to eliminate predictors? [duplicate]

I have 1 dependent variable and 33 independent variables (continuous, categorical & dichotomous). Correlation analyses (2-tailed) show that the DV is only correlated to 7 of the IVs although most ...
0
votes
1answer
40 views

Can a variable become statistically significant after the addition of another variable? [duplicate]

I am doing forward stepwise logistic regression. I have heard that its common for a previously statistically significant variable to become not statistically significant when one or more variables are ...
0
votes
0answers
29 views

Is it possible to use stepwise linear regression to find the parameters that explain the most variation in the output of a nonlinear system?

I have a mathematical model of a system. We have to use a simulation software to check the response of the system to specific input signals or change of model parameters. The relationship between some ...
0
votes
1answer
117 views

Controlling for age and sex in a multiple regression with a backward model selection

So, I have this dataset with a hormonal measure as independent variable and behavioral measures as dependent variables. I am using a linear regression as my model and after a backward selection, ended ...
0
votes
1answer
11 views

Detecting outlying distributions of ratio data

I have a dataset consisting of hundreds of repeat observations on thousands of agents. Each observation is a ratio between two distance measures, A and B, where A is always larger than B. Thus, my ...
1
vote
1answer
597 views

Combining principal component regression and stepwise regression

I want to use a combination of principal component analysis (PCA) and stepwise regression to develop a predictor model. I have 5 independent variables (which are correlated among each other to ...
0
votes
0answers
55 views

Variable reduction techniques

I am researching variable reduction techniques for time series data. Atm I came up with expert judgement, Stepwise Regression (Forward), Stepwise Regression (Backward) and Granger Causality. Any ...
2
votes
0answers
129 views

R: Dynamic Regression with ARIMA model using xreg, make use of step function?

This might fit better here than on stackoverflow, I guess. I was trying to build a dynamic regression model with the dynlm package, but it did not work out. After reading this by Hyndman, I now ...
0
votes
1answer
294 views

Significant predictors of mpg in mtcars dataset in R

Which variables are significant predictors of mpg in mtcars dataset. When I perform following regression: ...
2
votes
2answers
541 views

What is the difference between VIF and stepwise regression?

What is the difference between the variance inflation factor (VIF) and stepwise regression as both help in detecting multicollinearity? What variables are different while running both techniques?
0
votes
0answers
92 views

Do I drop insignificant parameters from a model? Should I use stepwise regression?

I'm working on a project where we have a number of factors we believe might have a role on a survey result. It's my job to figure out if this is true. My boss suggested just doing correlations on each ...
1
vote
0answers
174 views

R: Why does step function of a Linear Modegives different AIC/BIC than AIC function?

I don't understand what I make or think wrong, but if one tries to evaluate the linear model of the data (which you can find in R in the Package AIC(stats)), then ...
0
votes
0answers
384 views

How to use residual analysis to remove the effect of confounding variables in a model in R

I want to find which soil variables better explain plant productivity, using a database that contains information for about 100 forests plots across Europe. These plots have only one species per plot, ...
1
vote
1answer
280 views

Stepwise logistic regression

I am working with a dataset of 1000 individuals, 200 of which are disease positive. I have run a logistic regression with 25 predictors to identify overall which variables are significantly predictive....
2
votes
2answers
234 views

What is the best lag length for auto correlation?

I am doing a monthly rainfall forecasting model. I have monthly data from 1998 to 2012. I found in previous research that they have used partial autocorrelations and stepwise regression as an input ...
1
vote
1answer
207 views

Validity of stepwise regression in DistLM

I have a set of nutrient fluxes data and I would like to know which environmental drivers explains the fluxes. I used DistLM and the marginal test showed that none ...
2
votes
1answer
196 views

GAMLSS: model with interaction terms failed

I use gamlss method from library(gamlss) on my full models with interaction terms and try to reduce them with stepGAIC. There are 3 things I want to ask. Do I have to specify a link for the model? ...
4
votes
1answer
9k views

Stepwise Model Selection in Logistic Regression in R

I'm implementing a logistic regression model in R and I have 80 variables to chose from. I need to automatize the process of variable selection of the model so I'm using the step function. I've no ...