Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model.

1
vote
1answer
39 views

How to fit a stepwise regression with ARIMA errors using Arima function in R?

I am fitting a regression model with ARIMA errors in R using the Arima function from the ...
1
vote
1answer
27 views

feature importance using forward selection

In the following article the author has correctly mentioned that the "petal" is more important than "sepal" in case of iris data: https://towardsdatascience.com/feature-importance-and-forward-...
0
votes
0answers
19 views

Step-wise Multiple Regression or ANCOVA

I have an assignment that gives a dataset and a choice of 3 tests: Step-wise Multiple Regression, ANCOVA and Log-Linear Analysis. The dataset consists of ...
15
votes
1answer
508 views

Stepwise AIC - Does there exist controversy surrounding this topic?

I've read countless posts on this site that are incredibly against the use of stepwise selection of variables using any sort of criterion whether it be p-values based, AIC, BIC, etc. I understand why ...
1
vote
2answers
45 views

Significant main effects lost during ANCOVA due to interaction terms. Is type III the way to go?

I have some experimental data which I am analysing using step wise multiple regression (ANCOVA) in R using the step function. The response data (wp) is the leaf ...
0
votes
1answer
113 views

Backward elimination in a multinomial logistic regression model?

Following this UCLA article, I have fit a multinomial logistic regression model in R (say that Group is a factor with levels ...
4
votes
2answers
420 views

How to do stepwise regression with a binary dependent variable?

I want to use stepwise regression to reduce the number of variables. My dependent variable is a dummy variable (Fraud=1, None fraud=0) and I have 25 predictive variables. How can I do this?
0
votes
1answer
570 views

Stepwise regression with multinomial logit models in R

I'm trying to use the stepwise regression function step() on a mnlogit function of the ...
1
vote
1answer
109 views

forward model selection on multivariate polynomial regression with high dimension data

I am trying to fit the best multivariate polynomial on a dataset using stepAIC(). My problem is that I have more variables (p=3003) than observations (n=500), so ...
0
votes
1answer
32 views

Why does R step backwards regression drop variable with lowest AIC?

I'm running a backwards selection process in R using the step() function and it seems to be dropping variables based on lowest AIC associated with that variable. Is ...
0
votes
0answers
21 views

Reifying stepwise regression with additional predictors and hierarchical regression

I have performed (backwards elimination) stepwise regression using some fMRI data predictors to model spectroscopy data as a DV. This has resulted in some interesting models. I now have some ...
3
votes
1answer
39 views

Efficiently add a new predictor to an estimated multiple linear regression model

I want to use forward selection to choose predictors in a multiple linear regression model. If you have a regression with N predictors and want to add another predictor, is there a way to update the ...
0
votes
0answers
10 views

Model Selection with rates of change

I have a dataset with two regular predictor variables, x and y and two empirically estimated rates of change, dx and dy. I want to perform model selection in R (preferably using the ...
1
vote
1answer
37 views

Clarification-Forward stepwise regression

I'm learning about forward stepwise and there are some things which are not so clear: If I have $p$ predictors, is it true that forward stepwise does $p$ iterations? If I add the predictors in each ...
1
vote
0answers
35 views

How do you determine which “direction” you should go in a stepwise regression?

I realize that you can both go forward or backward, or even in both directions, however I'm finding it a little confusing when one is more appropriate than the other? Can somebody explain to me or ...
0
votes
1answer
53 views

Step-wise Regression with only Categorical Predictors

Suppose we are assessing the impact three factors, each with two levels, have on some response $Y$. Let's call the factors $A$ with levels $\{a_1, a_2\}$, $B$ with levels $\{b_1, b_2\}$and $C$ with ...
1
vote
1answer
272 views

Beta regression (betareg) with caret and train [closed]

I have a dataset with a dependent in range (0,1) and numerical/categorical predictors. Chiefly to streamline the code and easily accomplish cross validation (feature selection/model fitting), I would ...
0
votes
0answers
82 views

model selection and model comparison

I have a question regarding to model comparison using multcomp for model comparison. Suppose I have a linear model y~x1+x2+x3, and there are three levels in x1, say x1_l1, x1_l2 and x1_l3. I would ...
0
votes
0answers
379 views

Backward elimination for a non-linear multivariate regression

I'm trying to determine what would be a good model for my problem. I am not a statistician and use some words colloquially - please excuse my lack of knowledge. I'll illustrate the problem with the <...
1
vote
1answer
168 views

Hypothesis Testing on coefficients in two subsets of data after Stepwise Regression

Is it a reasonable approach to run a hypothesis test to test whether the coefficients of a variable in two regressions on two different subsets of the same population are different if you have used ...
0
votes
1answer
81 views

Variable selection in Hierarchical Linear Modelling HLM through nlme lme()

Background of my question:- In Linear Regression through R we can mention the direction="both"/"forward"/"backward" in step(lm()) function to tell R for choosing the best set of variables based on AIC....
0
votes
0answers
41 views

In using backward elimination procedure how to control for type I error?

Use backward elimination procedure to decide which predictor variables can be dropped from the regression model. Control the type I error at = . 10 at each stage In using backward ...
0
votes
0answers
50 views

Can nested cross validation combat collinearity in stepwise regression

UPDATE: I see now that is called nested cross validation. I simply wasn't aware of the procedure. So here's a simple question - will nested cross validation combat collinearity in a stepwise ...
1
vote
1answer
256 views

Why does forward stepwise selection reduce the AUC of a classifier to values < 0.500?

I've recently been benchmarking different methods for feature selection, and found a weird issue when using forward stepwise regression. Specifically, when I train a sparse logistic regression model ...
1
vote
1answer
579 views

Stepwise regression based on F-statistic in R [closed]

I know that the stepAIC function in R allows us to perform stepwise regression but I was wondering if there's any option (or other function) to perform a F-...
2
votes
1answer
3k views

Forward or backward sequential feature selection?

I was trying to carry out feature selection on a dataset using sequential feature selection. The dataset contains more than 5000 observations (rows) and 22 features (columns). Now I see that there are ...
2
votes
1answer
2k views

What is stepwise linear regression?

I am reading about 'interaction effects on linear regression' here and came across 'stepwise linear regression'. There are originally 5 predictors in the model. This means to say that by using the ...
0
votes
0answers
30 views

Stepwise not returning expected results in R

Why would step() output different outcomes when a better fit can be produced? I have two datasets model that should have the same relationships with a set of ...
2
votes
1answer
347 views

Recursive Feature Elimination in sklearn

I have been thinking about one thing after reading documentation from sklearn about Feature Selection for building prediction models (http://scikit-learn.org/stable/modules/feature_selection.html#rfe) ...
0
votes
0answers
1k views

What is the difference between Stepwise regression and Lasso regression in terms of variable selection?

What is the difference between Stepwise regression and Lasso regression in terms of variable selection? Is the difference just the way in which the variables are selected or is there any significant ...
0
votes
1answer
140 views

Methods of variable selection

A study stated that it used forward selection to chose variables for a multivariable regression model (in this case logistic) to evaluate association between predictor and outcome. They started with ...
0
votes
1answer
642 views

Simple linear regression vs. partial least squares (PLS)

I want to build a predictive model of an event in the spring based off of the weather during the winter (variable every year) and the soil characteristics (fixed) of many different sites. Although I ...
0
votes
1answer
143 views

Interpreting the order that predictor variables are added to a stepwise multiple linear regression model

Hi there I have currently been running a stepwise multiple linear regression in SPSS and have been having trouble interpreting the results. I have attached a link to the results below: Regression ...
2
votes
0answers
436 views

Are variable selection strategies for regression useful?

I have read many posts (including Frank Harrell's book) about the consequences of using variable selection strategies. However, it seems that many of the published work in the medical field still ...
1
vote
1answer
320 views

Stepwise quantile regression: What's the reason behind these strange results?

So I am attempting to build a model using quantile regression & am using stepwise regression for initial data exploration. I'm well aware that stepwise methods are widely frowned upon & am ...
0
votes
0answers
1k views

Error in solve.default(ddf) : Lapack routine dgesv: system is exactly singular: U[1,1] = 0

I am trying to use caret method called plr. I have installed stepPlr package. Here is how my code looks: ...
0
votes
1answer
46 views

Stepwise variable selection, significance and interpretations

I have read that the p-values of the variables resulting from a stepwise regression are smaller than it should be. So, suppose there are two independent individuals trying to address some problem. ...
6
votes
2answers
351 views

Do stepwise regression techniques increase a model's predictive power?

I understand some of the many problems of stepwise regression. However, as an academic endeavor, assume I want to use stepwise regression for a predictive model, and I want to better understand the ...
0
votes
1answer
40 views

After removing an outlier

I used step() in R to select a model. I found an data point with a high cook's distance and decided to remove it. After removing the outlier, should I used the current model or start again from the ...
4
votes
1answer
431 views

backwards stepwise regression, collinearity and regression to the mean

My research paper was recently rejected and some of the feedback I received was in relation to the statistical tests done/not done. I would like help in clarifying what I could do differently as the ...
0
votes
0answers
62 views

Why Stepwise regression lead to overfitting? [duplicate]

I run a rolling backward stepwise factor selection within each regression window with a Matrix of regressors X(137x481) and a vector Y(1x137). As you can see, the number of regressors is way higher ...
3
votes
1answer
256 views

Does it make sense to run a stepwise regression on components estimated through Partial Least Square?

I am trying to solve a problem of dimentionality reduction on a Matrix of predictors X(136x481). I found that PCA does not a good job in my case because it create components that explain just the ...
0
votes
0answers
80 views

Forward stepwise BIC selected a variable with Odds Ratio of 1

My purpose of running the a model is to give overview of what explains the outcome. I use BIC as criterion to run forward stepwise because BIC look for 'True' model. Prediction is not concern for me. ...
0
votes
1answer
119 views

Why stepwise feature selection method do not perform well when there is a large number of explanatory variables

I have the feeling that when I have a large number of predictors, it will be better to use feature selection regression model such as lasso, to fit the model, and better not use the stepwise feature ...
0
votes
1answer
65 views

Co-linearity/Stepwise regression

I was wondering if we need to check the multicollinearity between variables when we run the stepwise regression, If yes, why? Also, If we have several IV, how we should know which interaction we ...
0
votes
1answer
441 views

glm poisson regression - regressors selection

I have fit a GLM poisson regression model. Then i detected overdispersion, which was the reason that I have decided to fit a Negative Binomial model: ...
1
vote
1answer
70 views

Forward stepwise subset selection when number of predictors greater than number of instances

In the book Introduction to statistical learning(Section 6.1) it is mentioned that Forward stepwise selection can be applied even in the high-dimensional setting where n(instances) < p(predictors)....
0
votes
1answer
244 views

stepwise linear regression on a principal component

What I want to do I have 500,000 variables for a few hundreds records. I did principal component to find the subgroup of records. From the analysis I find that 4 principal components are sufficient ...
1
vote
0answers
170 views

Why avoid stepwise regression? [duplicate]

I have been using model averaging and model selection bases on AIC and BIC for a while. I have recently discover the stepwise regression technique and I found a lots of people critize this methods. ...