Questions tagged [stepwise-regression]
Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.
330
questions
0
votes
0
answers
17
views
Interpreting coefficients in Linear regression with categorical variables and one hot encoding (drop first)
I am doing multiple linear regression where my independent variables are a mix of categorical and numerical variables. Obviously I need to one-hot-encode the categorical variables, and I need to "...
2
votes
1
answer
32
views
Feature selection in a traditional regression model to an experiment data
I have an experiment data (total of 96) with 10 predictor and 2 response variables. I want to build a traditional multiple linear regression model to them in R. My aim is to build clearly ...
- 21
1
vote
0
answers
19
views
Automated Code for Logistic Regression [closed]
My Y variable (output) is binary (0 or 1). I have 10 input variables in total, 3 of them are scaled variable, 2 of them are ordinal number therefore being written with C( ). Rather than running the ...
- 11
1
vote
1
answer
84
views
How is missing values handled in a stepwise model process?
Suppose I have 100 observations among 4 variables, a Y with no missing values, then X1, X2, X3 that each have 10 (distinct) missing values, so that the complete case analysis has only N=70 ...
- 60.4k
1
vote
1
answer
186
views
The reason of different regression results between "enter" and "stepwise" methods
I and one of my colleagues conducted regression analysis in SPSS. There is a significant difference in our regression results obtained using the "enter" and "stepwise" methods. All ...
- 11
1
vote
0
answers
50
views
Using step() and car::vif(): order matters?
When fitting linear models and coming up with a plausible one, AIC and VIF are often used. However, I notice that the order in which the methods are used makes a difference on the final model.
Should ...
- 1,527
1
vote
0
answers
35
views
What relevance do p-values have in a multiple regression of all 50-states (a.k.a. population/census data)
I am running a regression analysis of the U.S. (specifically, the population represented in the U.S. Congress by voting members, so census data for the 50 states would constitute population data). One ...
- 11
1
vote
0
answers
36
views
Magnitude of Type I error inflation related to error noise after model selection
I am investigating Type I error inflation for the one-sided test $$H_0:\beta_2=0$$ after one step of forward stepwise regression for the following model: $$Y=X_1+\beta_2X_2+\epsilon$$ where $\epsilon \...
- 11
3
votes
1
answer
73
views
What does maintaining marginality mean in stepwise regression?
I was reading the helpfile for the dropterm function in the R MASS package, which is one of the main building blocks of the ...
- 4,070
1
vote
0
answers
14
views
Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R [closed]
Does anyone know how to model selection for function on function linear model to find the best subset of functional covariate in R
- 11
1
vote
0
answers
24
views
Selecting Predictors for an Outcome Model from a data set with Missing Values
I want to create a model to predict a numeric improvement (e.g. reduction in a numeric parameter) after surgery.
For that I want to choose the best predictors from a list of about 30 patient ...
- 11
5
votes
3
answers
639
views
How does cross validation works for feature selection (using stepwise regression)?
I have used the MATLAB regression learner application to do some stepwise regression with a 10-fold cross validation for feature selection. But now I want to code it myself and I'm confused about the ...
- 59
4
votes
2
answers
1k
views
Stepwise binary logistic regression, Forward or Backward?
I want to run a binary logistic regression to understanding (modeling) factors affecting nest-site selection in a bird species. I have Presence/Absence data and 13 predictors. The sample size is small....
- 139
0
votes
0
answers
34
views
Sampling and backwards selection
I'm working on a school project that involves performing backward stepwise regression as a form of feature selection. The dataset in question is 60k images with 700 total columns and is much too large ...
- 359
2
votes
0
answers
494
views
stepAIC in mixed models
As I haven't found the equivelant of the MASS::stepAIC for mixed models (eg in lmer) what I'm intending to do is to find the best lm model using stepAIC and then go in lmer and add the random effects.
...
- 347
3
votes
1
answer
251
views
F-statistic decreases when adding variables
When adding control variables to my regression, the F-statistic decreases. Furthermore, when I add an interaction term, the F-statistic is reduced further. How do I interpret these regression results?
- 31
0
votes
0
answers
26
views
Simpler test equivalent to shuffle tests and (doubly-nested) crossvalidation for the linear, Gaussian case?
My field often uses black-box models to predict a target $y \in \mathbb R^T$ from some covariates $x \in \mathbb R^{T\times N}$.
I'm using $T$ to denote the number of samples (presumed independent), ...
- 251
0
votes
1
answer
65
views
Can you use Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling?
I have a question regarding the use of Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling.
I am trying to perform Sequential Linear Modeling (with a binary outcome) ...
- 11
0
votes
0
answers
117
views
how to do stepwise regression with interaction terms?
I have a excel sheet with 100 independent variables and 1 dependent variable
Now I would like to run stepwise regression using SPSS with it considering pairwise interaction between them to.
As it may ...
- 59
3
votes
1
answer
219
views
A reasonable number of covariates after variable selection in a regression model
I read an unpublished paper. There is a regression model with about 20 covariates. The authors use a stepwise variable selection method and come to a model with two covariates with small p-values.
The ...
- 986
0
votes
0
answers
116
views
Online courses/resources for panel data analysis in python?
I am keen to learn more about panel data models using python, but I do not find related courses or many resources online. In particular, I usually find general statistics courses with Python or, ...
0
votes
0
answers
51
views
Starting point for backward selection in polynomial linear models in R
I am reading Generalized Additive Models by Simon N. Wood. I do not have a very strong formal background in maths and statistics, so this question might be too trivial. However, I am well versed in R.
...
- 101
2
votes
1
answer
49
views
Multiple regression with few observations and many variables
I have data about 40 stores described by 50+ continuous variables in terms of customer behaviour (types of purchases, demographic attributes, etc). I want to build a simple regression model to explain ...
- 125
5
votes
4
answers
209
views
Validity of regression diagnostics for deterministic computer experiments
The question
Is it possible to justify the use of regression diagnostics such as F-tests or the AIC for the analysis of deterministic computer experiments?
Background
One way to analyse computer ...
- 2,327
8
votes
1
answer
379
views
How competitive is stepwise regression when it comes to pure prediction?
When we want to do inference on parameters or nested models, stepwise variable selection causes a number of problems, discussed by Frank Harrell and others.
However, if we validate the stepwise model-...
- 58k
0
votes
0
answers
149
views
Not able to calculate BIC in bidirectional stepwise regression in R
I am trying to perform stepwise regression with direction as "both" based on BIC in r.
But everytime is performing based on AIC only
Following is my code
...
1
vote
1
answer
32
views
Which test would be best for my thesis study with 2 variables and 2 levels? I'm considering to run on SPSS, 4 separate correlation tests, OR a MLR
My thesis questions on whether a certain type of thoughts are associated with stress levels. There are two types of thoughts and two different levels. What should I use? The first variable is the ...
- 11
0
votes
0
answers
104
views
How can I force a predictor to stay in the model and select others using a stepwise method in a multiple linear regression in SPSS?
How can I force a predictor to stay in the model and select others using a stepwise method in a multiple linear regression in SPSS?
I would like to force one predictor to appear in my final model ...
1
vote
1
answer
168
views
effect sizes from stepwise regression
I'm new to stepwise regression and I've been asked to conduct one for my boss. In doing so, they also asked for the effect sizes from each predictor in the model. Disregarding any debates around ...
- 113
1
vote
0
answers
155
views
PCA, stepAIC, and negative binomial regression
I have some output data (around 800 data points) that very nicely fits a negative binomial distribution. I checked using fitdistr() in R and it is a very good fit.
Given this, my plan was to use ...
1
vote
0
answers
36
views
What is the difference between the criteria argument and the k= argument in step() in R?
What exactly is difference between the 'criteria' argument for the step() function in r thats described in the documentation here: https://search.r-project.org/CRAN/refmans/emdi/html/step.html
versus ...
- 11
0
votes
0
answers
24
views
Why odds ratios cannot be higher than the stepwise betas?
I am performing a logistic regression analysis after performing a stepwise regression.
I would like to know why the odds ratios of the logistic regression cannot be higher than the stepwise betas.
- 429
0
votes
0
answers
129
views
Reporting Stepwise Regression
G'day. For my research i calculated a backward stepwise regression for my first time using
...
1
vote
0
answers
53
views
Forward stepwise AIC approach
I am learning about performing stepwise model selection by AIC and I have 2 questions here:
1.Regarding to stepwise AIC, what is contribution (effectiveness) of the number of parameters in the model ...
- 51
1
vote
1
answer
281
views
Stepwise model selection by AIC
I am learning about performing stepwise model selection by AIC and having some questions:
What is the regularization parameter for step-AIC?
In what way is forward step-AIC an evolution of univariate ...
- 51
9
votes
3
answers
349
views
Should stepwise regressions also be avoided for exploratory (hypothesis generating) modelling?
In a recent paper, Andrew Tredennick and colleagues (2021) suggested to use the drop1() function in R for exploratory modelling (that is to generate new hypotheses ...
- 601
1
vote
1
answer
263
views
Does my predictor in my multiple regression have too many variables?
So I am trying to work out what is the best predictor of a) awareness over environmental issues, b) concern over environmental issues and c) pro-environmental behaviour from a set of sociodemographics ...
- 51
1
vote
0
answers
303
views
Can I use coefficients of ridge regression as feature importance values?
If I have normalized all predictors, can I use coefficients of ridge regression as feature importance values? Normalization in my case means:
$(x - u) / s$
where $u$ is the mean and $s$ is the ...
- 23
1
vote
0
answers
48
views
PCA to identify patterns in the data, forced to a particular variable?
Dataset: I have a hyperspectral dataset that consists 250 wavelength bands (x1,...x250) and corresponding reflectance measurements (y) for each band. Plotting X vs Y yields a spectral profile. I have ...
- 11
1
vote
0
answers
45
views
What am I missing to perform forward stepwise regression when p > n?
I keep running into warnings in RStudio when I use subsets where p > n. ISLR 6.4.3 mentions that forward stepwise can be useful for high dimensional data, which I'm trying to just test out for ...
- 59
0
votes
3
answers
118
views
How can I do Stepwise Method if only independent variables are given? [closed]
This is the data set
How can I determine if this 4 groups differed significantly in their performance
2
votes
1
answer
688
views
How to run backward elimination in $R$ with both categorical data and numeric data? [closed]
Usually in backward elimination, we start with the full model of all covariates, check the $p$-value of $t$-statistic for each covariate (which is compared between the full model and the model minus ...
- 285
0
votes
1
answer
240
views
Best-Subset Regression based on BIC versus Forward Selection based on AIC
I am trying to get a better grasp of BIC and AIC scores. I know BIC has a harsher penalty than AIC regarding model size (it prefers smaller, less complex models).
Suppose there is a situation where I ...
- 1
0
votes
0
answers
82
views
colinearity in stepwise regression
I would be very grateful if you can be so kind to clarify what may be wrong in the following steps and interpretation of the validity conditions of the stepwise regression.
I am afraid there is a ...
- 1
0
votes
1
answer
192
views
Regression on principal components (number of modes, evaluation, predictive value)
I seek advice on the use of regression on principal components.
I have a large dataset of anatomical shapes with point correspondence, on which I have performed PCA to investigate the main modes of ...
- 70
0
votes
1
answer
161
views
Is there a good replacement of sum of squared deviations that do not tend to split on edges?
I build a predictive model (regression) on a dataset that has just one real-valued feature and one real-valued target. To make it even simpler I want to find just a step function (decision tree with ...
- 554
2
votes
2
answers
65
views
What is the name of this algorithm for choosing which features to include in a neural network?
Once I read about one kind of neural network (for classification or feature selection) for a supervised training where you start with all inputs, then you proceed with a training step and randomly (or ...
- 163
0
votes
0
answers
259
views
leaps for stepwise selection: issues with factor variables
I'm trying to use the forward stepwise selection from the package leaps in order to choose the best predictors for a logistic regression.
This is the structure of my database
...
- 1
5
votes
1
answer
963
views
(Why) Are stepwise regression coefficients biased?
Issues with stepwise regression are known to statisticians.
It yields R-squared values that are badly biased to be high.
The F and chi-squared tests quoted next to each variable on the printout do ...
- 58k
0
votes
0
answers
518
views
panel data model and variable selection?
I am looking for a reference to perform variable selection with panel data.
I have 9 variables and want to test which variables to keep or remove. Someone recommended step-wise analysis, but I cannot ...
- 1