Skip to main content

Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

Filter by
Sorted by
Tagged with
0 votes
0 answers
40 views

Selecting degrees of freedom in stepwise regression (stepAIC function in R)

Context: I have data available on water quality in a number of catchments, for example the concentration of Zinc (Zn). For each catchment, I also have a range of characteristics (n=16), such as the ...
Matt Tomkins's user avatar
1 vote
0 answers
57 views

Least-bad stepwise procedure for a simulation that shows issues with stepwise regression

I am well-aware of the issues that stepwise regression causes. I want to demonstrate some of them via simulation in a particular situation. I am thinking of a regression where I have some categorical ...
Dave's user avatar
  • 65k
0 votes
0 answers
8 views

Can you deduce if a lasso model has a smaller/larger/equal RSS to a forward selection model?

I came across this question in my exam. Where there is a table where the columns are the different model selection methods: OLS, Lasso, Forward_Size1, ForwardSize2. And the rows are the predictors, ...
CodusOProgrammatus's user avatar
0 votes
0 answers
31 views

Can you infer that non-significant variables in full model won't be chosen by stepwise regression methods?

I recently encountered this question twice, on my exam. If you fit a full MLR additive, model, can you infer that the insignificant predictors (p-value > 0.05 from lm output) will not be chosen ...
CodusOProgrammatus's user avatar
0 votes
0 answers
20 views

Stepwise Regression - when will bi-direction give different results?

There is forward selection and backward elimination, and in both cases we can not only add or subtract variables, but also do both. My question is under which circumstances would the ...
Maverick Meerkat's user avatar
0 votes
2 answers
58 views

intuition linear regression stepwise selection of predictors

I am using a tool in genetics, which works very similar to stepwise linear regression (its called GCTA-COJO for those interested). Essentially the starting situation looks like this: You have 1000s of ...
nhaus's user avatar
  • 27
1 vote
1 answer
45 views

Significance test of an increase in adjusted R-squared between two models

I found two papers that provide their results showing the significance of an increase in adjusted R-squared between two models statistically, with p-values, to show the improvement after adding a few ...
J.K.'s user avatar
  • 143
0 votes
0 answers
103 views

Interpreting coefficients in Linear regression with categorical variables and one hot encoding (drop first)

I am doing multiple linear regression where my independent variables are a mix of categorical and numerical variables. Obviously I need to one-hot-encode the categorical variables, and I need to "...
jgklsdjfgkldsfaSDF's user avatar
2 votes
1 answer
44 views

Feature selection in a traditional regression model to an experiment data

I have an experiment data (total of 96) with 10 predictor and 2 response variables. I want to build a traditional multiple linear regression model to them in R. My aim is to build clearly ...
egeku's user avatar
  • 21
1 vote
0 answers
22 views

Automated Code for Logistic Regression [closed]

My Y variable (output) is binary (0 or 1). I have 10 input variables in total, 3 of them are scaled variable, 2 of them are ordinal number therefore being written with C( ). Rather than running the ...
sct 22n's user avatar
  • 11
2 votes
1 answer
293 views

How is missing values handled in a stepwise model process?

Suppose I have 100 observations among 4 variables, a Y with no missing values, then X1, X2, X3 that each have 10 (distinct) missing values, so that the complete case analysis has only N=70 ...
AdamO's user avatar
  • 63.7k
1 vote
1 answer
850 views

The reason of different regression results between "enter" and "stepwise" methods

I and one of my colleagues conducted regression analysis in SPSS. There is a significant difference in our regression results obtained using the "enter" and "stepwise" methods. All ...
Aurora Choi's user avatar
1 vote
0 answers
82 views

Using step() and car::vif(): order matters?

When fitting linear models and coming up with a plausible one, AIC and VIF are often used. However, I notice that the order in which the methods are used makes a difference on the final model. Should ...
compbiostats's user avatar
  • 1,579
1 vote
0 answers
44 views

What relevance do p-values have in a multiple regression of all 50-states (a.k.a. population/census data)

I am running a regression analysis of the U.S. (specifically, the population represented in the U.S. Congress by voting members, so census data for the 50 states would constitute population data). One ...
Sam Levi's user avatar
1 vote
0 answers
42 views

Magnitude of Type I error inflation related to error noise after model selection

I am investigating Type I error inflation for the one-sided test $$H_0:\beta_2=0$$ after one step of forward stepwise regression for the following model: $$Y=X_1+\beta_2X_2+\epsilon$$ where $\epsilon \...
Julia Mathis's user avatar
8 votes
1 answer
99 views

What does maintaining marginality mean in stepwise regression?

I was reading the helpfile for the dropterm function in the R MASS package, which is one of the main building blocks of the ...
Alex's user avatar
  • 4,452
1 vote
0 answers
16 views

Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R [closed]

Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R
Sky Sha's user avatar
  • 11
1 vote
0 answers
36 views

Selecting Predictors for an Outcome Model from a data set with Missing Values

I want to create a model to predict a numeric improvement (e.g. reduction in a numeric parameter) after surgery. For that I want to choose the best predictors from a list of about 30 patient ...
drwattson's user avatar
5 votes
3 answers
1k views

How does cross validation works for feature selection (using stepwise regression)?

I have used the MATLAB regression learner application to do some stepwise regression with a 10-fold cross validation for feature selection. But now I want to code it myself and I'm confused about the ...
Azarang's user avatar
  • 59
4 votes
2 answers
2k views

Stepwise binary logistic regression, Forward or Backward?

I want to run a binary logistic regression to understanding (modeling) factors affecting nest-site selection in a bird species. I have Presence/Absence data and 13 predictors. The sample size is small....
Mostafa Ahmadi's user avatar
0 votes
0 answers
44 views

Sampling and backwards selection

I'm working on a school project that involves performing backward stepwise regression as a form of feature selection. The dataset in question is 60k images with 700 total columns and is much too large ...
jmoore00's user avatar
  • 389
2 votes
0 answers
1k views

stepAIC in mixed models

As I haven't found the equivelant of the MASS::stepAIC for mixed models (eg in lmer) what I'm intending to do is to find the best lm model using stepAIC and then go in lmer and add the random effects. ...
Lefty's user avatar
  • 499
3 votes
1 answer
476 views

F-statistic decreases when adding variables

When adding control variables to my regression, the F-statistic decreases. Furthermore, when I add an interaction term, the F-statistic is reduced further. How do I interpret these regression results?
ines's user avatar
  • 31
0 votes
1 answer
81 views

Can you use Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling?

I have a question regarding the use of Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling. I am trying to perform Sequential Linear Modeling (with a binary outcome) ...
Sam16's user avatar
  • 11
0 votes
0 answers
259 views

how to do stepwise regression with interaction terms?

I have a excel sheet with 100 independent variables and 1 dependent variable Now I would like to run stepwise regression using SPSS with it considering pairwise interaction between them to. As it may ...
sriram's user avatar
  • 59
4 votes
1 answer
233 views

A reasonable number of covariates after variable selection in a regression model

I read an unpublished paper. There is a regression model with about 20 covariates. The authors use a stepwise variable selection method and come to a model with two covariates with small p-values. The ...
Viktor's user avatar
  • 1,035
0 votes
0 answers
60 views

Starting point for backward selection in polynomial linear models in R

I am reading Generalized Additive Models by Simon N. Wood. I do not have a very strong formal background in maths and statistics, so this question might be too trivial. However, I am well versed in R. ...
sp29's user avatar
  • 101
2 votes
1 answer
55 views

Multiple regression with few observations and many variables

I have data about 40 stores described by 50+ continuous variables in terms of customer behaviour (types of purchases, demographic attributes, etc). I want to build a simple regression model to explain ...
Strabonio's user avatar
  • 125
5 votes
4 answers
218 views

Validity of regression diagnostics for deterministic computer experiments

The question Is it possible to justify the use of regression diagnostics such as F-tests or the AIC for the analysis of deterministic computer experiments? Background One way to analyse computer ...
g g's user avatar
  • 2,708
8 votes
1 answer
484 views

How competitive is stepwise regression when it comes to pure prediction?

When we want to do inference on parameters or nested models, stepwise variable selection causes a number of problems, discussed by Frank Harrell and others. However, if we validate the stepwise model-...
Dave's user avatar
  • 65k
0 votes
0 answers
179 views

Not able to calculate BIC in bidirectional stepwise regression in R

I am trying to perform stepwise regression with direction as "both" based on BIC in r. But everytime is performing based on AIC only Following is my code ...
jigar somaiya's user avatar
1 vote
1 answer
37 views

Which test would be best for my thesis study with 2 variables and 2 levels? I'm considering to run on SPSS, 4 separate correlation tests, OR a MLR

My thesis questions on whether a certain type of thoughts are associated with stress levels. There are two types of thoughts and two different levels. What should I use? The first variable is the ...
Charmaine's user avatar
0 votes
0 answers
124 views

How can I force a predictor to stay in the model and select others using a stepwise method in a multiple linear regression in SPSS?

How can I force a predictor to stay in the model and select others using a stepwise method in a multiple linear regression in SPSS? I would like to force one predictor to appear in my final model ...
mariam alemairi's user avatar
1 vote
1 answer
223 views

effect sizes from stepwise regression

I'm new to stepwise regression and I've been asked to conduct one for my boss. In doing so, they also asked for the effect sizes from each predictor in the model. Disregarding any debates around ...
john connor's user avatar
1 vote
0 answers
193 views

PCA, stepAIC, and negative binomial regression

I have some output data (around 800 data points) that very nicely fits a negative binomial distribution. I checked using fitdistr() in R and it is a very good fit. Given this, my plan was to use ...
StatisticsPersonInTraining's user avatar
1 vote
0 answers
72 views

What is the difference between the criteria argument and the k= argument in step() in R?

What exactly is difference between the 'criteria' argument for the step() function in r thats described in the documentation here: https://search.r-project.org/CRAN/refmans/emdi/html/step.html versus ...
chris9759's user avatar
0 votes
0 answers
24 views

Why odds ratios cannot be higher than the stepwise betas?

I am performing a logistic regression analysis after performing a stepwise regression. I would like to know why the odds ratios of the logistic regression cannot be higher than the stepwise betas.
Adrián P.L.'s user avatar
1 vote
0 answers
62 views

Forward stepwise AIC approach

I am learning about performing stepwise model selection by AIC and I have 2 questions here: 1.Regarding to stepwise AIC, what is contribution (effectiveness) of the number of parameters in the model ...
tcxrp's user avatar
  • 51
1 vote
1 answer
405 views

Stepwise model selection by AIC

I am learning about performing stepwise model selection by AIC and having some questions: What is the regularization parameter for step-AIC? In what way is forward step-AIC an evolution of univariate ...
tcxrp's user avatar
  • 51
9 votes
3 answers
402 views

Should stepwise regressions also be avoided for exploratory (hypothesis generating) modelling?

In a recent paper, Andrew Tredennick and colleagues (2021) suggested to use the drop1() function in R for exploratory modelling (that is to generate new hypotheses ...
Fanfoué's user avatar
  • 641
1 vote
1 answer
387 views

Does my predictor in my multiple regression have too many variables?

So I am trying to work out what is the best predictor of a) awareness over environmental issues, b) concern over environmental issues and c) pro-environmental behaviour from a set of sociodemographics ...
Max Clarke's user avatar
1 vote
0 answers
311 views

Can I use coefficients of ridge regression as feature importance values?

If I have normalized all predictors, can I use coefficients of ridge regression as feature importance values? Normalization in my case means: $(x - u) / s$ where $u$ is the mean and $s$ is the ...
vldud's user avatar
  • 23
1 vote
0 answers
54 views

PCA to identify patterns in the data, forced to a particular variable?

Dataset: I have a hyperspectral dataset that consists 250 wavelength bands (x1,...x250) and corresponding reflectance measurements (y) for each band. Plotting X vs Y yields a spectral profile. I have ...
hcrawford's user avatar
1 vote
0 answers
47 views

What am I missing to perform forward stepwise regression when p > n?

I keep running into warnings in RStudio when I use subsets where p > n. ISLR 6.4.3 mentions that forward stepwise can be useful for high dimensional data, which I'm trying to just test out for ...
sumthymes's user avatar
0 votes
3 answers
131 views

How can I do Stepwise Method if only independent variables are given? [closed]

This is the data set How can I determine if this 4 groups differed significantly in their performance
Steven Alex's user avatar
2 votes
1 answer
884 views

How to run backward elimination in $R$ with both categorical data and numeric data? [closed]

Usually in backward elimination, we start with the full model of all covariates, check the $p$-value of $t$-statistic for each covariate (which is compared between the full model and the model minus ...
The One's user avatar
  • 225
0 votes
1 answer
424 views

Best-Subset Regression based on BIC versus Forward Selection based on AIC

I am trying to get a better grasp of BIC and AIC scores. I know BIC has a harsher penalty than AIC regarding model size (it prefers smaller, less complex models). Suppose there is a situation where I ...
Autumn's user avatar
  • 1
0 votes
0 answers
95 views

colinearity in stepwise regression

I would be very grateful if you can be so kind to clarify what may be wrong in the following steps and interpretation of the validity conditions of the stepwise regression. I am afraid there is a ...
LeoFer's user avatar
  • 1
0 votes
1 answer
223 views

Regression on principal components (number of modes, evaluation, predictive value)

I seek advice on the use of regression on principal components. I have a large dataset of anatomical shapes with point correspondence, on which I have performed PCA to investigate the main modes of ...
Niels's user avatar
  • 170
0 votes
1 answer
164 views

Is there a good replacement of sum of squared deviations that do not tend to split on edges?

I build a predictive model (regression) on a dataset that has just one real-valued feature and one real-valued target. To make it even simpler I want to find just a step function (decision tree with ...
Roman's user avatar
  • 612

1
2 3 4 5
7