Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

Filter by
Sorted by
Tagged with
0
votes
1answer
32 views

How to deal with predictors which are not significant, although r-squared is significant?

I did factor analysis and found three factors. To examine if which factors significantly affected a certain dependent variable, I added all three factors to a regression model. The correlation ...
3
votes
2answers
32 views

R: Multiple Linear Regression - Prediction model with 4 Indenpendent Continuous Variables

We are tasked to build a linear model to predict the current of the river based on river width, river depth, distance to the ocean and bank height in order to understand the variation in stream ...
3
votes
2answers
101 views

Significance of epidemiological confounders in a generalized linear model

I am identifying risk factors for children snoring among several predictors with generalized linear model. With backward selection, age and sex do not appear to be significant so I removed them from ...
0
votes
0answers
11 views

Stepwise feature selection for a classifier: use accuracy or loss? [duplicate]

When using stepwise feature selection for a classifier, is it typical to measure progress by using the loss (e.g., cross-entropy loss) or the accuracy? Is there any basis for choosing one or the ...
0
votes
0answers
46 views

Backward stepwise selection stopping rule in an ordinal logistic regression context when a model's ranking ability is of importance

First off, I am aware that there are some problems with stepwise regression as for instance described here ;) I am saying this to avoid that the discussion goes in the direction of stepwise being an ...
2
votes
1answer
54 views

Forward and backward stepwise regression (AIC) for negative binomial regression (with real data)

I am doing some count data analysis. The data is in this link. Column A is the count data, and other columns are the independent variables. At first I used Poisson regression to analyze it: ...
2
votes
1answer
61 views

Understanding 'aggressiveness' of lasso, forward stepwise selection and best subset selection in Hastie, T., Tibshirani, R. & Tibshirani, R.J. (2017)

Hastie et al. (2017) explain how the above mentioned methods perform depending on the signal-to-noise ratio (SNR) with their varying 'aggressiveness'. Now I don't understand why the different methods ...
0
votes
0answers
19 views

Using stepAIC to help select final model [duplicate]

So I have a full model such as; ...
0
votes
0answers
26 views

Best subset regression with instrumental variable

I am applying multiple regression with a data. There are 19 regressors in total and one of them is endogenous. For the endogenous variable I have identified an instrumental variable. When I apply ...
0
votes
1answer
29 views

Is forward selection using AIC as selection critiria valid? [duplicate]

I'm using a sequential approach to decide the best fitting model for my data. (I'm still new to R, so I decided to go for a manual approach rather than an automated one offered by R packages). I'm ...
0
votes
1answer
45 views

In hierarchical regression, the first step is not significant but later steps are

I ran a hierarchical regression comparing different models, in order of complexity. The first step was not significant (the simplest model vs a more complex model). However the next two steps were ...
0
votes
1answer
142 views

Stepwise regression - what are the steps in STATA?

This question will probably seem very stupid, but hey, econometrics and statistics were never really my strongest features! For my BA, my professor adviced me to perform stepwise regression. My ...
4
votes
2answers
117 views

Multiple regression with more than 20 predictors

I have a numerical dependent variable, and many independent variables. Most of my independent variables are dummy variables, but I have some categorical and numerical variables, too. I tried forward ...
0
votes
0answers
6 views

When building a multiple linear regression model, is it possible to form models with both linear and non-linear (quadratic) relationships?

Through backward elimination, I have reduced my model from 6 linear factors to 1, accounting for 68% of variance. I have also found that by squaring one of the variables I previously included, that ...
0
votes
0answers
27 views

How to group factor levels for stepwise regression using caret

Using the train() function from caret in R, I'm trying to run a stepwise ANCOVA, but each level of my 9-level factor is being ...
0
votes
0answers
786 views

warning message glm.fit fitted probabilities numerically 0 or 1 occurred in r

I was faced with the following error as I fitted a logistic regression. warning message glm.fit fitted probabilities numerically 0 or 1 occurred in r I searched for the right answer in the community,...
0
votes
0answers
14 views

Selecting variables using forward selection method

If we are to select predictors for a regression model using forward selection and the information available to us is just a correlation table, do we select the predictors that have a strong ...
0
votes
0answers
18 views

what types of regression is appropriate when independent variables are normally distributed but the dependent variable is not normal?

I have five variables (all continuous), in which the twtwo are normally distributed and three are non-normal (according to my preliminary results). But I want to conduct multiple regression in the ...
0
votes
0answers
4 views

Moving Beyond 'Proc-Fish' and stepwise model building

So stepwise is bad. Can anyone point me to a resource on how I ought build models, an online one? (I know there are books out there on this, but I don't have any handy right now...)
0
votes
0answers
59 views

Stepwise regression in R - what's my alternate?

Details I'm building what is called a direct demand model for predicting boardings at rail transit stations. The most available example is Transit Cooperative Research Project report 16 (TCRP 16). I ...
0
votes
0answers
9 views

Are lower order terms less likely to be removed from the model?

I have a model $$y=x_0+x_1+x_2+x_0x_1+x_1x_2+x_0x_2+x_1x_2x_3$$, which is saturated, I am trying to remove terms which are found to be insignificant. I understand that if, say $x_1$ was removed, then ...
0
votes
0answers
9 views

How to specify the number of output variables in step() stepwise selection in R? [closed]

I want to find important variables. Thus, I want the step() result to contain 20 variables, but it contains about 40 variables. Can I write some codes to accomplish this?
0
votes
0answers
18 views

Correct interpretation of forward stepwise model selection output

I have a question about my interpretation of stepwise model selection, but first let me explain my data: I have some data on the number of parasites that are counted on fish certain distances away ...
1
vote
0answers
16 views

Is there the equivalent of the stepAIC function for PERMANOVA?

I am comparing the presence and abundance of species between different sites, which are different in more than 1 factor, and the factors are not independent. I am looking to get the relative ...
0
votes
0answers
27 views

Stepwise regression for left-censored using NADA - R

I'm working with environmental data which are left-censored and I found the R package NADA which seems to do the job. After fitting a complete model, using the cenreg function,I'd like to do a ...
0
votes
0answers
36 views

Stepwise Regression

I have some data that I want to fit a log-linear model to (using R). The data in this dataset is categorical and y is the frequency. I use the glm function with family=poisson. I firstly fit the ...
0
votes
0answers
16 views

How to fit the maximal model when not enough observations?

I am trying to model a dataset with GLMs but I am wondering how to start with the first step of fitting a maximal model that tests all covariates and their interactions when there are simply not ...
4
votes
1answer
335 views

RFE vs Backward Elimination - is there a difference?

I recently discovered the RFE tool, and love it. I'd like to understand how this is different from vanilla backward elimination. Despite lots of information about these two techniques, the penny ...
1
vote
2answers
395 views

Feature subset selection by stepwise regression for a random forest model?

I would like to build a random forest model for regression. I have an abundance of potential features, and I expect only some of them to have a significant impact on the target variable. In addition, ...
0
votes
0answers
25 views

how to choose performance metrics when forward selecting in logistic regression?

I am new to statistics. I am performing a multi-class logistic regression and I want to select the important features. So I am implementing a forward selection. So, first I normalised the data and ...
2
votes
0answers
10 views

Best way to determine how the answers from a survey correspond to each other?

I conducted a survey on about 200 people and would like to know if there are any patterns in how people responded. To be more specific, is there a correlation between how people responded to different ...
0
votes
0answers
42 views

Best Feature selection algorithm Boruta, Step, Information Values(WoE) or RFE

I have landing data with 103 columns Would like to understand which algorithm tis best for feature selection and what may be the logic to call any feature as best. I have landing data with 103 ...
0
votes
2answers
23 views

How to directly know the backward selection model when independent variables are orthogonal?

According to this output, the independent variables are orthogonal. Please tell me, when doing the backward selection, why it can be directly known that it should be reduced to 5th order model?
0
votes
0answers
26 views

What do I do if the variable with the highest variance inflation factor is a single power term with higher order terms present?

I am trying to fix multicollinearity in a linear regression model using stepwise (backward elimination) variable selection and variance inflation factors (VIFs). Let's say I have a model that is ...
1
vote
0answers
65 views

How to exploit orthogonalization procedure in forward stepwise linear regression?

Forward Stepwise linear regression allows to build up a subset of features starting from the intercept. At each step the predictor that most improves the fit is added to the subset. In the book, it ...
0
votes
0answers
69 views

Ordinal regression, categorical variables, and “step” function

I am doing an ordinal regression analysis using "polr" function. I got a result of the regression analysis and continued to use "step" function to find the final prediction model. As all my variables ...
1
vote
1answer
197 views

Is the final model of backward elimination with AIC ​always​ the same as the final model of forward elimination with AIC?

I have a question: is the final model of backward elimination with AIC ​always​ the same as the final model of forward elimination with AIC? I assume that it is the same result
0
votes
1answer
31 views

Regression analysis not showing the first level of treatments

I have my data that looks like this: data: or in R: ...
1
vote
0answers
67 views

How does stepwise ARIMA work?

How does a stepwise ARIMA model work? I understand how ARIMA works but i didn't find any good material to understand about stepwise ARIMA. Any leads will be helpful. Thanks
0
votes
1answer
193 views

Feature selection (backward elimination) in polynomial regression

I have a polynomial multiple (univariate) regression with 2nd degree (for example) as below. Question. When I execute backward elimination to select features, should I remove features from the ...
1
vote
1answer
60 views

When should you check if assumptions are met when using stepwise selection?

Suppose I want to find a linear model with Gaussian error for a given data set. (The data set contains insurance claims and the end goal is to predict claim cost from claim features.) Also, suppose ...
1
vote
0answers
38 views

My predictors are all categorical variables but the dependent is numerical, how to eliminate dummies?

My predictors are all categorical values but the dependent is numerical. How can I eliminate dummies if I use a linear regression model? The values are tough to solve with backward elimination; ...
1
vote
1answer
103 views

In stepwise regression, how to interpret non-significant variables? [duplicate]

I have more than 15 IVs such as age, gender, education, first language, technology proficiency, health condition, etc, and one of my DVs is health literacy level, which is measured through a standard ...
1
vote
0answers
985 views

How exactly does Bidirectional (Stepwise) elimination works?

I read about Forward selection and Backward elimination algorithms while learning to build machine learning models. I think I'm not quite clear on next approach, which was bidirectional elimination(...
8
votes
0answers
142 views

Understanding equation used by Hastie et al

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. I am not clear on what the equation on the Y-axis means ...
0
votes
1answer
1k views

forward selection with mixed model using lmer [closed]

I am running a mixed model in R and would like to perform forward selection using the step function. However, when I set the direction to forward ...
6
votes
0answers
266 views

Recreating figure from Elements of Statistical Learning [closed]

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. To recreate the forward stepwise line my process is as ...
20
votes
2answers
551 views

Does LASSO suffer from the same problems stepwise regression does?

Stepwise algorithmic variable-selection methods tend to select for models which bias more or less every estimate in regression models ($\beta$s and their SEs, p-values, F statistics, etc.), and are ...
1
vote
0answers
36 views

Equivalence of variable selection criteria in forward stepwise regression

Say we have already selected $x_1$ through $x_k$. To select the next variable, forward stepwise regression either: a. picks the variable that when added to the already selected variables, gives ...
0
votes
0answers
25 views

How to choose the independent variables in a GLMM without performing stepwise selection? With a global model? How to decide then?

I am trying to conduct an inferential binomial GLMM with a large dataset and many independent variables. I was attempting to do a stepwise AIC selection but keep reading it is a bad idea. However, ...

1
2 3 4 5 6