Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

232 questions
Filter by
Sorted by
Tagged with
29 views

Regression analysis not showing the first level of treatments

I have my data that looks like this: data: or in R: ...
27 views

How does stepwise ARIMA work?

How does a stepwise ARIMA model work? I understand how ARIMA works but i didn't find any good material to understand about stepwise ARIMA. Any leads will be helpful. Thanks
28 views

Feature selection (backward elimination) in polynomial regression

I have a polynomial multiple (univariate) regression with 2nd degree (for example) as below. Question. When I execute backward elimination to select features, should I remove features from the ...
45 views

When should you check if assumptions are met when using stepwise selection?

Suppose I want to find a linear model with Gaussian error for a given data set. (The data set contains insurance claims and the end goal is to predict claim cost from claim features.) Also, suppose ...
31 views

My predictors are all categorical variables but the dependent is numerical, how to eliminate dummies?

My predictors are all categorical values but the dependent is numerical. How can I eliminate dummies if I use a linear regression model? The values are tough to solve with backward elimination; ...
43 views

In stepwise regression, how to interpret non-significant variables? [duplicate]

I have more than 15 IVs such as age, gender, education, first language, technology proficiency, health condition, etc, and one of my DVs is health literacy level, which is measured through a standard ...
84 views

How exactly does Bidirectional (Stepwise) elimination works?

I read about Forward selection and Backward elimination algorithms while learning to build machine learning models. I think I'm not quite clear on next approach, which was bidirectional elimination(...
10 views

Test Error on various types of Regression

I'm testing a dataset for various types of regression, comparing test error for each one to the Mean Prediction Error, that I found at the beginning. Unfortunately I don't have any experience in this ...
116 views

Understanding equation used by Hastie et al

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. I am not clear on what the equation on the Y-axis means ...
109 views

forward selection with mixed model using lmer [closed]

I am running a mixed model in R and would like to perform forward selection using the step function. However, when I set the direction to forward ...
216 views

Recreating figure from Elements of Statistical Learning [closed]

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. To recreate the forward stepwise line my process is as ...
326 views

Does LASSO suffer from the same problems stepwise regression does?

Stepwise algorithmic variable-selection methods tend to select for models which bias more or less every estimate in regression models ($\beta$s and their SEs, p-values, F statistics, etc.), and are ...
27 views

Equivalence of variable selection criteria in forward stepwise regression

Say we have already selected $x_1$ through $x_k$. To select the next variable, forward stepwise regression either: a. picks the variable that when added to the already selected variables, gives ...
19 views

How to choose the independent variables in a GLMM without performing stepwise selection? With a global model? How to decide then?

I am trying to conduct an inferential binomial GLMM with a large dataset and many independent variables. I was attempting to do a stepwise AIC selection but keep reading it is a bad idea. However, ...
83 views

Stepwise regression limitations avoided on bootstrap/independent datasets?

One of the prime objections to best-subset and stepwise regression techniques (forward selection and/or backward elimination) is that multiple hypothesis tests are conducted on the same dataset, ...
26 views

Time series novel

I've exhaustively attempted to find a proper way to analyse a dataset. Despite finding several piece of information of what could be done, I kindly ask for suggestions of could be done, mainly in R. ...
9 views

Is there a good plug and play method for online/iterative regression in scikit-learn?

I have an independent variable with 19 dimensions(19 features) and I need to perform stepwise regression. I need to perform iterative regression because the target value I am predicting becomes ...
28 views

Linear Regression in stats model OLS

After removing the insignificant variables(p-value >.05), I fitted the OLS model again. I found there are still many variables which had p-value < .05 earlier have p-value > .05 now. Do I need to ...
36 views

Backward and forward selection finds insignificant predictors

I have a set of possible predictors for a binary outcome. In order to obtain the best model, I start from the zero model, and do a stepwise selection (in R) in order to obtain the best predictors. The ...
22 views

44 views

Should I report the pseudo $R^2$ value for full or final logistic regression model after removing NA's & running stepwise selection?

I'm working with a logistic regression model in r. model <- glm(response~., family="binomial", data) and I'm using ...
34 views

Stepwise logistic regression: What exactly is meant by eliminating features based on contribution?

I would like to know how my program is selecting and removing features during stepwise regression. I'm using R's caret package which in turn I think is using stepAIC from the MASS package. I was ...
42 views

Equivalence between Stepwise Regression and Lasso

A while ago I had learned of a theoretical result that suggested there was a correspondence with Forward/Backward/Stepwise regression and LASSO/RIDGE regression in terms of the coefficient of ...
344 views

How to fit a stepwise regression with ARIMA errors using Arima function in R?

I am fitting a regression model with ARIMA errors in R using the Arima function from the ...
116 views

feature importance using forward selection

In the following article the author has correctly mentioned that the "petal" is more important than "sepal" in case of iris data: https://towardsdatascience.com/feature-importance-and-forward-...
40 views

Step-wise Multiple Regression or ANCOVA

I have an assignment that gives a dataset and a choice of 3 tests: Step-wise Multiple Regression, ANCOVA and Log-Linear Analysis. The dataset consists of ...
708 views

Stepwise AIC - Does there exist controversy surrounding this topic?

I've read countless posts on this site that are incredibly against the use of stepwise selection of variables using any sort of criterion whether it be p-values based, AIC, BIC, etc. I understand why ...
59 views

Significant main effects lost during ANCOVA due to interaction terms. Is type III the way to go?

I have some experimental data which I am analysing using step wise multiple regression (ANCOVA) in R using the step function. The response data (wp) is the leaf ...
402 views

Backward elimination in a multinomial logistic regression model?

Following this UCLA article, I have fit a multinomial logistic regression model in R (say that Group is a factor with levels ...
1k views

How to do stepwise regression with a binary dependent variable?

I want to use stepwise regression to reduce the number of variables. My dependent variable is a dummy variable (Fraud=1, None fraud=0) and I have 25 predictive variables. How can I do this?
407 views

forward model selection on multivariate polynomial regression with high dimension data

I am trying to fit the best multivariate polynomial on a dataset using stepAIC(). My problem is that I have more variables (p=3003) than observations (n=500), so ...
63 views

Why does R step backwards regression drop variable with lowest AIC?

I'm running a backwards selection process in R using the step() function and it seems to be dropping variables based on lowest AIC associated with that variable. Is ...
57 views

Efficiently add a new predictor to an estimated multiple linear regression model

I want to use forward selection to choose predictors in a multiple linear regression model. If you have a regression with N predictors and want to add another predictor, is there a way to update the ...
62 views

Clarification-Forward stepwise regression

I'm learning about forward stepwise and there are some things which are not so clear: If I have $p$ predictors, is it true that forward stepwise does $p$ iterations? If I add the predictors in each ...
42 views

How do you determine which “direction” you should go in a stepwise regression?

I realize that you can both go forward or backward, or even in both directions, however I'm finding it a little confusing when one is more appropriate than the other? Can somebody explain to me or ...
152 views

Metrics for selecting a logistic regression model

I have been running logistic regressions using forward, backward and 'both direction' stepwise procedures to guide the selection of the variables included in the model. I have been using AIC as a ...
63 views

Step-wise Regression with only Categorical Predictors

Suppose we are assessing the impact three factors, each with two levels, have on some response $Y$. Let's call the factors $A$ with levels $\{a_1, a_2\}$, $B$ with levels $\{b_1, b_2\}$and $C$ with ...
542 views

Beta regression (betareg) with caret and train [closed]

I have a dataset with a dependent in range (0,1) and numerical/categorical predictors. Chiefly to streamline the code and easily accomplish cross validation (feature selection/model fitting), I would ...
92 views

model selection and model comparison

I have a question regarding to model comparison using multcomp for model comparison. Suppose I have a linear model y~x1+x2+x3, and there are three levels in x1, say x1_l1, x1_l2 and x1_l3. I would ...
697 views

Backward elimination for a non-linear multivariate regression

I'm trying to determine what would be a good model for my problem. I am not a statistician and use some words colloquially - please excuse my lack of knowledge. I'll illustrate the problem with the <...
253 views

Hypothesis Testing on coefficients in two subsets of data after Stepwise Regression

Is it a reasonable approach to run a hypothesis test to test whether the coefficients of a variable in two regressions on two different subsets of the same population are different if you have used ...
134 views

Variable selection in Hierarchical Linear Modelling HLM through nlme lme()

Background of my question:- In Linear Regression through R we can mention the direction="both"/"forward"/"backward" in step(lm()) function to tell R for choosing the best set of variables based on AIC....
85 views

In using backward elimination procedure how to control for type I error?

Use backward elimination procedure to decide which predictor variables can be dropped from the regression model. Control the type I error at = . 10 at each stage In using backward ...
474 views

Why does forward stepwise selection reduce the AUC of a classifier to values < 0.500?

I've recently been benchmarking different methods for feature selection, and found a weird issue when using forward stepwise regression. Specifically, when I train a sparse logistic regression model ...
1k views

Stepwise regression based on F-statistic in R [closed]

I know that the stepAIC function in R allows us to perform stepwise regression but I was wondering if there's any option (or other function) to perform a F-...
5k views

Forward or backward sequential feature selection?

I was trying to carry out feature selection on a dataset using sequential feature selection. The dataset contains more than 5000 observations (rows) and 22 features (columns). Now I see that there are ...