Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

0
votes
0answers
16 views

How to choose the independent variables in a GLMM without performing stepwise selection? With a global model? How to decide then?

I am trying to conduct an inferential binomial GLMM with a large dataset and many independent variables. I was attempting to do a stepwise AIC selection but keep reading it is a bad idea. However, ...
2
votes
1answer
44 views

Stepwise regression limitations avoided on bootstrap/independent datasets?

One of the prime objections to best-subset and stepwise regression techniques (forward selection and/or backward elimination) is that multiple hypothesis tests are conducted on the same dataset, ...
0
votes
0answers
21 views

Time series novel

I've exhaustively attempted to find a proper way to analyse a dataset. Despite finding several piece of information of what could be done, I kindly ask for suggestions of could be done, mainly in R. ...
0
votes
0answers
8 views

Is there a good plug and play method for online/iterative regression in scikit-learn?

I have an independent variable with 19 dimensions(19 features) and I need to perform stepwise regression. I need to perform iterative regression because the target value I am predicting becomes ...
2
votes
1answer
24 views

Linear Regression in stats model OLS

After removing the insignificant variables(p-value >.05), I fitted the OLS model again. I found there are still many variables which had p-value < .05 earlier have p-value > .05 now. Do I need to ...
1
vote
1answer
29 views

Backward and forward selection finds insignificant predictors

I have a set of possible predictors for a binary outcome. In order to obtain the best model, I start from the zero model, and do a stepwise selection (in R) in order to obtain the best predictors. The ...
0
votes
1answer
18 views

Confusing stepwise regression process

According to the algorithm for the backward stepwise selection from the book ISLR which is shown below: says that we need to choose the model among the $k$ models by having a smallest RSS or highest $...
0
votes
0answers
24 views

does stepwise regression only work when there are a few explanatory variables with a significant correlation with the dependent variable?

I understand that stepwise regression is computationally intensive in general but is it only "suitable" in cases where you can ignore several variables from the model due to statistical insignificance,...
0
votes
3answers
58 views

Lasso Regression as Variable Selection

Suppose we are initially given $p$ predictor variables. In lasso regression, we want to find estimates of the coefficients $\beta_1, \dots, \beta_p$ that minimize $\text{RSS}+ \lambda \sum_{j=1}^{p} |\...
1
vote
1answer
25 views

Should I report the pseudo $R^2$ value for full or final logistic regression model after removing NA's & running stepwise selection?

I'm working with a logistic regression model in r. model <- glm(response~., family="binomial", data) and I'm using ...
1
vote
0answers
26 views

Stepwise logistic regression: What exactly is meant by eliminating features based on contribution?

I would like to know how my program is selecting and removing features during stepwise regression. I'm using R's caret package which in turn I think is using stepAIC from the MASS package. I was ...
0
votes
0answers
25 views

Equivalence between Stepwise Regression and Lasso

A while ago I had learned of a theoretical result that suggested there was a correspondence with Forward/Backward/Stepwise regression and LASSO/RIDGE regression in terms of the coefficient of ...
1
vote
1answer
149 views

How to fit a stepwise regression with ARIMA errors using Arima function in R?

I am fitting a regression model with ARIMA errors in R using the Arima function from the ...
1
vote
1answer
45 views

feature importance using forward selection

In the following article the author has correctly mentioned that the "petal" is more important than "sepal" in case of iris data: https://towardsdatascience.com/feature-importance-and-forward-...
0
votes
0answers
27 views

Step-wise Multiple Regression or ANCOVA

I have an assignment that gives a dataset and a choice of 3 tests: Step-wise Multiple Regression, ANCOVA and Log-Linear Analysis. The dataset consists of ...
17
votes
1answer
595 views

Stepwise AIC - Does there exist controversy surrounding this topic?

I've read countless posts on this site that are incredibly against the use of stepwise selection of variables using any sort of criterion whether it be p-values based, AIC, BIC, etc. I understand why ...
1
vote
2answers
55 views

Significant main effects lost during ANCOVA due to interaction terms. Is type III the way to go?

I have some experimental data which I am analysing using step wise multiple regression (ANCOVA) in R using the step function. The response data (wp) is the leaf ...
0
votes
1answer
269 views

Backward elimination in a multinomial logistic regression model?

Following this UCLA article, I have fit a multinomial logistic regression model in R (say that Group is a factor with levels ...
4
votes
2answers
825 views

How to do stepwise regression with a binary dependent variable?

I want to use stepwise regression to reduce the number of variables. My dependent variable is a dummy variable (Fraud=1, None fraud=0) and I have 25 predictive variables. How can I do this?
0
votes
2answers
1k views

Stepwise regression with multinomial logit models in R

I'm trying to use the stepwise regression function step() on a mnlogit function of the ...
1
vote
1answer
272 views

forward model selection on multivariate polynomial regression with high dimension data

I am trying to fit the best multivariate polynomial on a dataset using stepAIC(). My problem is that I have more variables (p=3003) than observations (n=500), so ...
0
votes
1answer
47 views

Why does R step backwards regression drop variable with lowest AIC?

I'm running a backwards selection process in R using the step() function and it seems to be dropping variables based on lowest AIC associated with that variable. Is ...
0
votes
0answers
23 views

Reifying stepwise regression with additional predictors and hierarchical regression

I have performed (backwards elimination) stepwise regression using some fMRI data predictors to model spectroscopy data as a DV. This has resulted in some interesting models. I now have some ...
3
votes
1answer
46 views

Efficiently add a new predictor to an estimated multiple linear regression model

I want to use forward selection to choose predictors in a multiple linear regression model. If you have a regression with N predictors and want to add another predictor, is there a way to update the ...
0
votes
0answers
12 views

Model Selection with rates of change

I have a dataset with two regular predictor variables, x and y and two empirically estimated rates of change, dx and dy. I want to perform model selection in R (preferably using the ...
1
vote
1answer
47 views

Clarification-Forward stepwise regression

I'm learning about forward stepwise and there are some things which are not so clear: If I have $p$ predictors, is it true that forward stepwise does $p$ iterations? If I add the predictors in each ...
1
vote
0answers
39 views

How do you determine which “direction” you should go in a stepwise regression?

I realize that you can both go forward or backward, or even in both directions, however I'm finding it a little confusing when one is more appropriate than the other? Can somebody explain to me or ...
0
votes
1answer
57 views

Step-wise Regression with only Categorical Predictors

Suppose we are assessing the impact three factors, each with two levels, have on some response $Y$. Let's call the factors $A$ with levels $\{a_1, a_2\}$, $B$ with levels $\{b_1, b_2\}$and $C$ with ...
1
vote
1answer
401 views

Beta regression (betareg) with caret and train [closed]

I have a dataset with a dependent in range (0,1) and numerical/categorical predictors. Chiefly to streamline the code and easily accomplish cross validation (feature selection/model fitting), I would ...
0
votes
0answers
86 views

model selection and model comparison

I have a question regarding to model comparison using multcomp for model comparison. Suppose I have a linear model y~x1+x2+x3, and there are three levels in x1, say x1_l1, x1_l2 and x1_l3. I would ...
0
votes
0answers
503 views

Backward elimination for a non-linear multivariate regression

I'm trying to determine what would be a good model for my problem. I am not a statistician and use some words colloquially - please excuse my lack of knowledge. I'll illustrate the problem with the <...
1
vote
1answer
215 views

Hypothesis Testing on coefficients in two subsets of data after Stepwise Regression

Is it a reasonable approach to run a hypothesis test to test whether the coefficients of a variable in two regressions on two different subsets of the same population are different if you have used ...
0
votes
1answer
111 views

Variable selection in Hierarchical Linear Modelling HLM through nlme lme()

Background of my question:- In Linear Regression through R we can mention the direction="both"/"forward"/"backward" in step(lm()) function to tell R for choosing the best set of variables based on AIC....
0
votes
0answers
68 views

In using backward elimination procedure how to control for type I error?

Use backward elimination procedure to decide which predictor variables can be dropped from the regression model. Control the type I error at = . 10 at each stage In using backward ...
1
vote
1answer
367 views

Why does forward stepwise selection reduce the AUC of a classifier to values < 0.500?

I've recently been benchmarking different methods for feature selection, and found a weird issue when using forward stepwise regression. Specifically, when I train a sparse logistic regression model ...
1
vote
1answer
842 views

Stepwise regression based on F-statistic in R [closed]

I know that the stepAIC function in R allows us to perform stepwise regression but I was wondering if there's any option (or other function) to perform a F-...
2
votes
1answer
4k views

Forward or backward sequential feature selection?

I was trying to carry out feature selection on a dataset using sequential feature selection. The dataset contains more than 5000 observations (rows) and 22 features (columns). Now I see that there are ...
3
votes
1answer
3k views

What is stepwise linear regression?

I am reading about 'interaction effects on linear regression' here and came across 'stepwise linear regression'. There are originally 5 predictors in the model. This means to say that by using the ...
0
votes
0answers
32 views

Stepwise not returning expected results in R

Why would step() output different outcomes when a better fit can be produced? I have two datasets model that should have the same relationships with a set of ...
2
votes
1answer
399 views

Recursive Feature Elimination in sklearn

I have been thinking about one thing after reading documentation from sklearn about Feature Selection for building prediction models (http://scikit-learn.org/stable/modules/feature_selection.html#rfe) ...
0
votes
0answers
1k views

What is the difference between Stepwise regression and Lasso regression in terms of variable selection?

What is the difference between Stepwise regression and Lasso regression in terms of variable selection? Is the difference just the way in which the variables are selected or is there any significant ...
0
votes
1answer
189 views

Methods of variable selection

A study stated that it used forward selection to chose variables for a multivariable regression model (in this case logistic) to evaluate association between predictor and outcome. They started with ...
0
votes
1answer
846 views

Simple linear regression vs. partial least squares (PLS)

I want to build a predictive model of an event in the spring based off of the weather during the winter (variable every year) and the soil characteristics (fixed) of many different sites. Although I ...
0
votes
1answer
198 views

Interpreting the order that predictor variables are added to a stepwise multiple linear regression model

Hi there I have currently been running a stepwise multiple linear regression in SPSS and have been having trouble interpreting the results. I have attached a link to the results below: Regression ...
2
votes
0answers
510 views

Are variable selection strategies for regression useful?

I have read many posts (including Frank Harrell's book) about the consequences of using variable selection strategies. However, it seems that many of the published work in the medical field still ...
1
vote
1answer
382 views

Stepwise quantile regression: What's the reason behind these strange results?

So I am attempting to build a model using quantile regression & am using stepwise regression for initial data exploration. I'm well aware that stepwise methods are widely frowned upon & am ...
0
votes
1answer
47 views

Stepwise variable selection, significance and interpretations

I have read that the p-values of the variables resulting from a stepwise regression are smaller than it should be. So, suppose there are two independent individuals trying to address some problem. ...
8
votes
2answers
457 views

Do stepwise regression techniques increase a model's predictive power?

I understand some of the many problems of stepwise regression. However, as an academic endeavor, assume I want to use stepwise regression for a predictive model, and I want to better understand the ...
0
votes
1answer
53 views

After removing an outlier

I used step() in R to select a model. I found an data point with a high cook's distance and decided to remove it. After removing the outlier, should I used the current model or start again from the ...
4
votes
1answer
554 views

backwards stepwise regression, collinearity and regression to the mean

My research paper was recently rejected and some of the feedback I received was in relation to the statistical tests done/not done. I would like help in clarifying what I could do differently as the ...