# Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

267 questions
Filter by
Sorted by
Tagged with
32 views

### How to deal with predictors which are not significant, although r-squared is significant?

I did factor analysis and found three factors. To examine if which factors significantly affected a certain dependent variable, I added all three factors to a regression model. The correlation ...
32 views

### R: Multiple Linear Regression - Prediction model with 4 Indenpendent Continuous Variables

We are tasked to build a linear model to predict the current of the river based on river width, river depth, distance to the ocean and bank height in order to understand the variation in stream ...
101 views

### Significance of epidemiological confounders in a generalized linear model

I am identifying risk factors for children snoring among several predictors with generalized linear model. With backward selection, age and sex do not appear to be significant so I removed them from ...
11 views

### Stepwise feature selection for a classifier: use accuracy or loss? [duplicate]

When using stepwise feature selection for a classifier, is it typical to measure progress by using the loss (e.g., cross-entropy loss) or the accuracy? Is there any basis for choosing one or the ...
46 views

### Backward stepwise selection stopping rule in an ordinal logistic regression context when a model's ranking ability is of importance

First off, I am aware that there are some problems with stepwise regression as for instance described here ;) I am saying this to avoid that the discussion goes in the direction of stepwise being an ...
54 views

### Forward and backward stepwise regression (AIC) for negative binomial regression (with real data)

I am doing some count data analysis. The data is in this link. Column A is the count data, and other columns are the independent variables. At first I used Poisson regression to analyze it: ...
61 views

### Understanding 'aggressiveness' of lasso, forward stepwise selection and best subset selection in Hastie, T., Tibshirani, R. & Tibshirani, R.J. (2017)

Hastie et al. (2017) explain how the above mentioned methods perform depending on the signal-to-noise ratio (SNR) with their varying 'aggressiveness'. Now I don't understand why the different methods ...
19 views

### Using stepAIC to help select final model [duplicate]

So I have a full model such as; ...
26 views

### Best subset regression with instrumental variable

I am applying multiple regression with a data. There are 19 regressors in total and one of them is endogenous. For the endogenous variable I have identified an instrumental variable. When I apply ...
29 views

### Is forward selection using AIC as selection critiria valid? [duplicate]

I'm using a sequential approach to decide the best fitting model for my data. (I'm still new to R, so I decided to go for a manual approach rather than an automated one offered by R packages). I'm ...
45 views

### In hierarchical regression, the first step is not significant but later steps are

I ran a hierarchical regression comparing different models, in order of complexity. The first step was not significant (the simplest model vs a more complex model). However the next two steps were ...
142 views

### Stepwise regression - what are the steps in STATA?

This question will probably seem very stupid, but hey, econometrics and statistics were never really my strongest features! For my BA, my professor adviced me to perform stepwise regression. My ...
117 views

### Multiple regression with more than 20 predictors

I have a numerical dependent variable, and many independent variables. Most of my independent variables are dummy variables, but I have some categorical and numerical variables, too. I tried forward ...
6 views

### When building a multiple linear regression model, is it possible to form models with both linear and non-linear (quadratic) relationships?

Through backward elimination, I have reduced my model from 6 linear factors to 1, accounting for 68% of variance. I have also found that by squaring one of the variables I previously included, that ...
27 views

### How to group factor levels for stepwise regression using caret

Using the train() function from caret in R, I'm trying to run a stepwise ANCOVA, but each level of my 9-level factor is being ...
786 views

### warning message glm.fit fitted probabilities numerically 0 or 1 occurred in r

I was faced with the following error as I fitted a logistic regression. warning message glm.fit fitted probabilities numerically 0 or 1 occurred in r I searched for the right answer in the community,...
14 views

### Selecting variables using forward selection method

If we are to select predictors for a regression model using forward selection and the information available to us is just a correlation table, do we select the predictors that have a strong ...
18 views

### what types of regression is appropriate when independent variables are normally distributed but the dependent variable is not normal?

I have five variables (all continuous), in which the twtwo are normally distributed and three are non-normal (according to my preliminary results). But I want to conduct multiple regression in the ...
4 views

### Moving Beyond 'Proc-Fish' and stepwise model building

So stepwise is bad. Can anyone point me to a resource on how I ought build models, an online one? (I know there are books out there on this, but I don't have any handy right now...)
59 views

### Stepwise regression in R - what's my alternate?

Details I'm building what is called a direct demand model for predicting boardings at rail transit stations. The most available example is Transit Cooperative Research Project report 16 (TCRP 16). I ...
9 views

### Are lower order terms less likely to be removed from the model?

I have a model $$y=x_0+x_1+x_2+x_0x_1+x_1x_2+x_0x_2+x_1x_2x_3$$, which is saturated, I am trying to remove terms which are found to be insignificant. I understand that if, say $x_1$ was removed, then ...
9 views

### How to specify the number of output variables in step() stepwise selection in R? [closed]

I want to find important variables. Thus, I want the step() result to contain 20 variables, but it contains about 40 variables. Can I write some codes to accomplish this?
18 views

### Correct interpretation of forward stepwise model selection output

I have a question about my interpretation of stepwise model selection, but first let me explain my data: I have some data on the number of parasites that are counted on fish certain distances away ...
16 views

### Is there the equivalent of the stepAIC function for PERMANOVA?

I am comparing the presence and abundance of species between different sites, which are different in more than 1 factor, and the factors are not independent. I am looking to get the relative ...
27 views

### Stepwise regression for left-censored using NADA - R

I'm working with environmental data which are left-censored and I found the R package NADA which seems to do the job. After fitting a complete model, using the cenreg function,I'd like to do a ...
36 views

### Stepwise Regression

I have some data that I want to fit a log-linear model to (using R). The data in this dataset is categorical and y is the frequency. I use the glm function with family=poisson. I firstly fit the ...
16 views

### How to fit the maximal model when not enough observations?

I am trying to model a dataset with GLMs but I am wondering how to start with the first step of fitting a maximal model that tests all covariates and their interactions when there are simply not ...
335 views

### RFE vs Backward Elimination - is there a difference?

I recently discovered the RFE tool, and love it. I'd like to understand how this is different from vanilla backward elimination. Despite lots of information about these two techniques, the penny ...
395 views

### Feature subset selection by stepwise regression for a random forest model?

I would like to build a random forest model for regression. I have an abundance of potential features, and I expect only some of them to have a significant impact on the target variable. In addition, ...
25 views

### how to choose performance metrics when forward selecting in logistic regression?

I am new to statistics. I am performing a multi-class logistic regression and I want to select the important features. So I am implementing a forward selection. So, first I normalised the data and ...
10 views

### Best way to determine how the answers from a survey correspond to each other?

I conducted a survey on about 200 people and would like to know if there are any patterns in how people responded. To be more specific, is there a correlation between how people responded to different ...
42 views

### Best Feature selection algorithm Boruta, Step, Information Values(WoE) or RFE

I have landing data with 103 columns Would like to understand which algorithm tis best for feature selection and what may be the logic to call any feature as best. I have landing data with 103 ...
23 views

### How to directly know the backward selection model when independent variables are orthogonal?

According to this output, the independent variables are orthogonal. Please tell me, when doing the backward selection, why it can be directly known that it should be reduced to 5th order model?
26 views

### What do I do if the variable with the highest variance inflation factor is a single power term with higher order terms present?

I am trying to fix multicollinearity in a linear regression model using stepwise (backward elimination) variable selection and variance inflation factors (VIFs). Let's say I have a model that is ...
65 views

### How to exploit orthogonalization procedure in forward stepwise linear regression?

Forward Stepwise linear regression allows to build up a subset of features starting from the intercept. At each step the predictor that most improves the fit is added to the subset. In the book, it ...
69 views

### Ordinal regression, categorical variables, and “step” function

I am doing an ordinal regression analysis using "polr" function. I got a result of the regression analysis and continued to use "step" function to find the final prediction model. As all my variables ...
197 views

### Is the final model of backward elimination with AIC ​always​ the same as the final model of forward elimination with AIC?

I have a question: is the final model of backward elimination with AIC ​always​ the same as the final model of forward elimination with AIC? I assume that it is the same result
31 views

### Regression analysis not showing the first level of treatments

I have my data that looks like this: data: or in R: ...
67 views

### How does stepwise ARIMA work?

How does a stepwise ARIMA model work? I understand how ARIMA works but i didn't find any good material to understand about stepwise ARIMA. Any leads will be helpful. Thanks
193 views

### Feature selection (backward elimination) in polynomial regression

I have a polynomial multiple (univariate) regression with 2nd degree (for example) as below. Question. When I execute backward elimination to select features, should I remove features from the ...
60 views

### When should you check if assumptions are met when using stepwise selection?

Suppose I want to find a linear model with Gaussian error for a given data set. (The data set contains insurance claims and the end goal is to predict claim cost from claim features.) Also, suppose ...
38 views

### My predictors are all categorical variables but the dependent is numerical, how to eliminate dummies?

My predictors are all categorical values but the dependent is numerical. How can I eliminate dummies if I use a linear regression model? The values are tough to solve with backward elimination; ...
103 views

### In stepwise regression, how to interpret non-significant variables? [duplicate]

I have more than 15 IVs such as age, gender, education, first language, technology proficiency, health condition, etc, and one of my DVs is health literacy level, which is measured through a standard ...
985 views

### How exactly does Bidirectional (Stepwise) elimination works?

I read about Forward selection and Backward elimination algorithms while learning to build machine learning models. I think I'm not quite clear on next approach, which was bidirectional elimination(...
142 views

### Understanding equation used by Hastie et al

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. I am not clear on what the equation on the Y-axis means ...
1k views

### forward selection with mixed model using lmer [closed]

I am running a mixed model in R and would like to perform forward selection using the step function. However, when I set the direction to forward ...
266 views

### Recreating figure from Elements of Statistical Learning [closed]

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. To recreate the forward stepwise line my process is as ...
551 views

### Does LASSO suffer from the same problems stepwise regression does?

Stepwise algorithmic variable-selection methods tend to select for models which bias more or less every estimate in regression models ($\beta$s and their SEs, p-values, F statistics, etc.), and are ...
Say we have already selected $x_1$ through $x_k$. To select the next variable, forward stepwise regression either: a. picks the variable that when added to the already selected variables, gives ...