Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

When building a multiple linear regression model, is it possible to form models with both linear and non-linear (quadratic) relationships?

Through backward elimination, I have reduced my model from 6 linear factors to 1, accounting for 68% of variance. I have also found that by squaring one of the variables I previously included, that ...
0
votes
0answers
18 views

How to group factor levels for stepwise regression using caret

Using the train() function from caret in R, I'm trying to run a stepwise ANCOVA, but each level of my 9-level factor is being ...
0
votes
0answers
183 views

warning message glm.fit fitted probabilities numerically 0 or 1 occurred in r

I was faced with the following error as I fitted a logistic regression. warning message glm.fit fitted probabilities numerically 0 or 1 occurred in r I searched for the right answer in the community,...
0
votes
0answers
13 views

Selecting variables using forward selection method

If we are to select predictors for a regression model using forward selection and the information available to us is just a correlation table, do we select the predictors that have a strong ...
0
votes
0answers
16 views

what types of regression is appropriate when independent variables are normally distributed but the dependent variable is not normal?

I have five variables (all continuous), in which the twtwo are normally distributed and three are non-normal (according to my preliminary results). But I want to conduct multiple regression in the ...
0
votes
0answers
3 views

Moving Beyond 'Proc-Fish' and stepwise model building

So stepwise is bad. Can anyone point me to a resource on how I ought build models, an online one? (I know there are books out there on this, but I don't have any handy right now...)
0
votes
0answers
44 views

Stepwise regression in R - what's my alternate?

Details I'm building what is called a direct demand model for predicting boardings at rail transit stations. The most available example is Transit Cooperative Research Project report 16 (TCRP 16). I ...
0
votes
0answers
8 views

Are lower order terms less likely to be removed from the model?

I have a model $$y=x_0+x_1+x_2+x_0x_1+x_1x_2+x_0x_2+x_1x_2x_3$$, which is saturated, I am trying to remove terms which are found to be insignificant. I understand that if, say $x_1$ was removed, then ...
0
votes
0answers
3 views

How to specify the number of output variables in step() stepwise selection in R?

I want to find important variables. Thus, I want the step() result to contain 20 variables, but it contains about 40 variables. Can I write some codes to accomplish this?
0
votes
0answers
17 views

Correct interpretation of forward stepwise model selection output

I have a question about my interpretation of stepwise model selection, but first let me explain my data: I have some data on the number of parasites that are counted on fish certain distances away ...
0
votes
0answers
14 views

Why forward and backward stepwise selection do not always give same predictors?

Forward and backward stepwise selection method do not always give exactly same predictors, even with same number of terms included. Why does this happen? In what case these two methods will always (or ...
1
vote
0answers
12 views

Is there the equivalent of the stepAIC function for PERMANOVA?

I am comparing the presence and abundance of species between different sites, which are different in more than 1 factor, and the factors are not independent. I am looking to get the relative ...
0
votes
0answers
15 views

Stepwise regression for left-censored using NADA - R

I'm working with environmental data which are left-censored and I found the R package NADA which seems to do the job. After fitting a complete model, using the cenreg function,I'd like to do a ...
0
votes
0answers
35 views

Stepwise Regression

I have some data that I want to fit a log-linear model to (using R). The data in this dataset is categorical and y is the frequency. I use the glm function with family=poisson. I firstly fit the ...
0
votes
0answers
8 views

How to fit the maximal model when not enough observations?

I am trying to model a dataset with GLMs but I am wondering how to start with the first step of fitting a maximal model that tests all covariates and their interactions when there are simply not ...
4
votes
1answer
124 views

RFE vs Backward Elimination - is there a difference?

I recently discovered the RFE tool, and love it. I'd like to understand how this is different from vanilla backward elimination. Despite lots of information about these two techniques, the penny ...
0
votes
2answers
170 views

Feature subset selection by stepwise regression for a random forest model?

I would like to build a random forest model for regression. I have an abundance of potential features, and I expect only some of them to have a significant impact on the target variable. In addition, ...
0
votes
0answers
23 views

how to choose performance metrics when forward selecting in logistic regression?

I am new to statistics. I am performing a multi-class logistic regression and I want to select the important features. So I am implementing a forward selection. So, first I normalised the data and ...
2
votes
0answers
10 views

Best way to determine how the answers from a survey correspond to each other?

I conducted a survey on about 200 people and would like to know if there are any patterns in how people responded. To be more specific, is there a correlation between how people responded to different ...
0
votes
0answers
32 views

Best Feature selection algorithm Boruta, Step, Information Values(WoE) or RFE

I have landing data with 103 columns Would like to understand which algorithm tis best for feature selection and what may be the logic to call any feature as best. I have landing data with 103 ...
0
votes
2answers
22 views

How to directly know the backward selection model when independent variables are orthogonal?

According to this output, the independent variables are orthogonal. Please tell me, when doing the backward selection, why it can be directly known that it should be reduced to 5th order model?
0
votes
0answers
21 views

What do I do if the variable with the highest variance inflation factor is a single power term with higher order terms present?

I am trying to fix multicollinearity in a linear regression model using stepwise (backward elimination) variable selection and variance inflation factors (VIFs). Let's say I have a model that is ...
1
vote
0answers
46 views

How to exploit orthogonalization procedure in forward stepwise linear regression?

Forward Stepwise linear regression allows to build up a subset of features starting from the intercept. At each step the predictor that most improves the fit is added to the subset. In the book, it ...
0
votes
0answers
49 views

Ordinal regression, categorical variables, and “step” function

I am doing an ordinal regression analysis using "polr" function. I got a result of the regression analysis and continued to use "step" function to find the final prediction model. As all my variables ...
1
vote
1answer
112 views

Is the final model of backward elimination with AIC ​always​ the same as the final model of forward elimination with AIC?

I have a question: is the final model of backward elimination with AIC ​always​ the same as the final model of forward elimination with AIC? I assume that it is the same result
0
votes
1answer
30 views

Regression analysis not showing the first level of treatments

I have my data that looks like this: data: or in R: ...
1
vote
0answers
37 views

How does stepwise ARIMA work?

How does a stepwise ARIMA model work? I understand how ARIMA works but i didn't find any good material to understand about stepwise ARIMA. Any leads will be helpful. Thanks
0
votes
1answer
113 views

Feature selection (backward elimination) in polynomial regression

I have a polynomial multiple (univariate) regression with 2nd degree (for example) as below. Question. When I execute backward elimination to select features, should I remove features from the ...
1
vote
1answer
55 views

When should you check if assumptions are met when using stepwise selection?

Suppose I want to find a linear model with Gaussian error for a given data set. (The data set contains insurance claims and the end goal is to predict claim cost from claim features.) Also, suppose ...
1
vote
0answers
37 views

My predictors are all categorical variables but the dependent is numerical, how to eliminate dummies?

My predictors are all categorical values but the dependent is numerical. How can I eliminate dummies if I use a linear regression model? The values are tough to solve with backward elimination; ...
1
vote
1answer
66 views

In stepwise regression, how to interpret non-significant variables? [duplicate]

I have more than 15 IVs such as age, gender, education, first language, technology proficiency, health condition, etc, and one of my DVs is health literacy level, which is measured through a standard ...
1
vote
0answers
681 views

How exactly does Bidirectional (Stepwise) elimination works?

I read about Forward selection and Backward elimination algorithms while learning to build machine learning models. I think I'm not quite clear on next approach, which was bidirectional elimination(...
0
votes
0answers
14 views

Test Error on various types of Regression

I'm testing a dataset for various types of regression, comparing test error for each one to the Mean Prediction Error, that I found at the beginning. Unfortunately I don't have any experience in this ...
7
votes
0answers
132 views

Understanding equation used by Hastie et al

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. I am not clear on what the equation on the Y-axis means ...
0
votes
1answer
646 views

forward selection with mixed model using lmer [closed]

I am running a mixed model in R and would like to perform forward selection using the step function. However, when I set the direction to forward ...
6
votes
0answers
244 views

Recreating figure from Elements of Statistical Learning [closed]

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. To recreate the forward stepwise line my process is as ...
20
votes
2answers
477 views

Does LASSO suffer from the same problems stepwise regression does?

Stepwise algorithmic variable-selection methods tend to select for models which bias more or less every estimate in regression models ($\beta$s and their SEs, p-values, F statistics, etc.), and are ...
1
vote
0answers
30 views

Equivalence of variable selection criteria in forward stepwise regression

Say we have already selected $x_1$ through $x_k$. To select the next variable, forward stepwise regression either: a. picks the variable that when added to the already selected variables, gives ...
0
votes
0answers
22 views

How to choose the independent variables in a GLMM without performing stepwise selection? With a global model? How to decide then?

I am trying to conduct an inferential binomial GLMM with a large dataset and many independent variables. I was attempting to do a stepwise AIC selection but keep reading it is a bad idea. However, ...
3
votes
1answer
249 views

Stepwise regression limitations avoided on bootstrap/independent datasets?

One of the prime objections to best-subset and stepwise regression techniques (forward selection and/or backward elimination) is that multiple hypothesis tests are conducted on the same dataset, ...
0
votes
0answers
11 views

Is there a good plug and play method for online/iterative regression in scikit-learn?

I have an independent variable with 19 dimensions(19 features) and I need to perform stepwise regression. I need to perform iterative regression because the target value I am predicting becomes ...
2
votes
1answer
31 views

Linear Regression in stats model OLS

After removing the insignificant variables(p-value >.05), I fitted the OLS model again. I found there are still many variables which had p-value < .05 earlier have p-value > .05 now. Do I need to ...
1
vote
1answer
53 views

Backward and forward selection finds insignificant predictors

I have a set of possible predictors for a binary outcome. In order to obtain the best model, I start from the zero model, and do a stepwise selection (in R) in order to obtain the best predictors. The ...
0
votes
1answer
31 views

Confusing stepwise regression process

According to the algorithm for the backward stepwise selection from the book ISLR which is shown below: says that we need to choose the model among the $k$ models by having a smallest RSS or highest $...
0
votes
0answers
24 views

does stepwise regression only work when there are a few explanatory variables with a significant correlation with the dependent variable?

I understand that stepwise regression is computationally intensive in general but is it only "suitable" in cases where you can ignore several variables from the model due to statistical insignificance,...
1
vote
4answers
351 views

Lasso Regression as Variable Selection

Suppose we are initially given $p$ predictor variables. In lasso regression, we want to find estimates of the coefficients $\beta_1, \dots, \beta_p$ that minimize $\text{RSS}+ \lambda \sum_{j=1}^{p} |\...
1
vote
1answer
100 views

Should I report the pseudo $R^2$ value for full or final logistic regression model after removing NA's & running stepwise selection?

I'm working with a logistic regression model in r. model <- glm(response~., family="binomial", data) and I'm using ...
1
vote
0answers
41 views

Stepwise logistic regression: What exactly is meant by eliminating features based on contribution?

I would like to know how my program is selecting and removing features during stepwise regression. I'm using R's caret package which in turn I think is using stepAIC from the MASS package. I was ...
0
votes
0answers
58 views

Equivalence between Stepwise Regression and Lasso

A while ago I had learned of a theoretical result that suggested there was a correspondence with Forward/Backward/Stepwise regression and LASSO/RIDGE regression in terms of the coefficient of ...
1
vote
1answer
619 views

How to fit a stepwise regression with ARIMA errors using Arima function in R?

I am fitting a regression model with ARIMA errors in R using the Arima function from the ...

1
2 3 4 5 6