Questions tagged [stepwise-regression]

Stepwise regression (often called forward or backward regression) involves fitting a regression model and adding or removing predictors based on $t$ statistics, $R^2$ or information criteria to arrive in a *stepwise* manner at a final model. This tag can also be used for forward selection, backward elimination & best subsets variable selection strategies.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

forward stepwise AIC approach

I am learning about performing stepwise model selection by AIC and I have 2 questions here: 1.Regarding to stepwise AIC, what is contribution (effectiveness) of the number of parameters in the model ...
user avatar
  • 41
1 vote
1 answer
23 views

Stepwise model selection by AIC

I am learning about performing stepwise model selection by AIC and having some questions: What is the regularization parameter for step-AIC? In what way is forward step-AIC an evolution of univariate ...
user avatar
  • 41
0 votes
0 answers
25 views

Manual variable selection in GLMM

I'm modeling my data using GLMM with 1 random factor and 10 variables that are of interest. Instead of using automatic selection, I started with the full model (including all variables except for the ...
user avatar
0 votes
0 answers
15 views

How does lmStepAIC work in caret when using cross-validation?

In caret package in R , one can train linear models with stepwise selection based on AIC, using this function : ...
user avatar
  • 253
0 votes
0 answers
25 views

In Variable selection method, should we include all the variables?

I've been working on a class project. While building a model, We've selected a few initial predictors (8 predictors out of 20) based on Business Knowledge. Next, we wanted to choose predictors based ...
user avatar
3 votes
1 answer
70 views

Should stepwise regressions or overfitting also be avoided for exploratory (hypothesis generating) modelling?

In a recent paper, Andrew Tredennick and colleagues (2021) suggested to use the drop1() function in R for exploratory modelling (that is to generate new hypotheses ...
user avatar
  • 383
1 vote
1 answer
63 views

Does my predictor in my multiple regression have too many variables?

So I am trying to work out what is the best predictor of a) awareness over environmental issues, b) concern over environmental issues and c) pro-environmental behaviour from a set of sociodemographics ...
user avatar
1 vote
0 answers
99 views

Can I use coefficients of ridge regression as feature importance values?

If I have normalized all predictors, can I use coefficients of ridge regression as feature importance values? Normalization in my case means: $(x - u) / s$ where $u$ is the mean and $s$ is the ...
user avatar
  • 13
1 vote
0 answers
15 views

PCA to identify patterns in the data, forced to a particular variable?

Dataset: I have a hyperspectral dataset that consists 250 wavelength bands (x1,...x250) and corresponding reflectance measurements (y) for each band. Plotting X vs Y yields a spectral profile. I have ...
user avatar
1 vote
0 answers
37 views

What am I missing to perform forward stepwise regression when p > n?

I keep running into warnings in RStudio when I use subsets where p > n. ISLR 6.4.3 mentions that forward stepwise can be useful for high dimensional data, which I'm trying to just test out for ...
user avatar
0 votes
3 answers
77 views

How can I do Stepwise Method if only independent variables are given? [closed]

This is the data set How can I determine if this 4 groups differed significantly in their performance
user avatar
2 votes
1 answer
67 views

How to run backward elimination in $R$ with both categorical data and numeric data? [closed]

Usually in backward elimination, we start with the full model of all covariates, check the $p$-value of $t$-statistic for each covariate (which is compared between the full model and the model minus ...
user avatar
  • 143
0 votes
1 answer
40 views

Best-Subset Regression based on BIC versus Forward Selection based on AIC

I am trying to get a better grasp of BIC and AIC scores. I know BIC has a harsher penalty than AIC regarding model size (it prefers smaller, less complex models). Suppose there is a situation where I ...
user avatar
  • 1
0 votes
0 answers
41 views

colinearity in stepwise regression

I would be very grateful if you can be so kind to clarify what may be wrong in the following steps and interpretation of the validity conditions of the stepwise regression. I am afraid there is a ...
user avatar
  • 1
0 votes
1 answer
69 views

Regression on principal components (number of modes, evaluation, predictive value)

I seek advice on the use of regression on principal components. I have a large dataset of anatomical shapes with point correspondence, on which I have performed PCA to investigate the main modes of ...
user avatar
  • 60
0 votes
0 answers
24 views

Interpreting beta estimates of categorical variable in stepwise forward selection in r

I am recruiting individuals based on their spends in a large retail store. 60 features are considered altogether and many are categorical type with many levels. State is one such variable with 27 ...
user avatar
  • 203
0 votes
1 answer
98 views

Is there a good replacement of sum of squared deviations that do not tend to split on edges?

I build a predictive model (regression) on a dataset that has just one real-valued feature and one real-valued target. To make it even simpler I want to find just a step function (decision tree with ...
user avatar
  • 463
2 votes
2 answers
46 views

What is the name of this algorithm for choosing which features to include in a neural network?

Once I read about one kind of neural network (for classification or feature selection) for a supervised training where you start with all inputs, then you proceed with a training step and randomly (or ...
user avatar
  • 143
0 votes
0 answers
37 views

leaps for stepwise selection: issues with factor variables

I'm trying to use the forward stepwise selection from the package leaps in order to choose the best predictors for a logistic regression. This is the structure of my database ...
user avatar
  • 1
0 votes
0 answers
28 views

Regarding stepwise and hierarchical regression (regression model building)

I am currently analysing a dataset and I was wondering if it is wrong to use a stepwise regression (to reduce the number of predictors) followed by a hierarchical regression (add demographics to the ...
user avatar
  • 1
0 votes
0 answers
20 views

feature selection by considering a reference variable before clustering

I want to cluster 39 homes based on 22 features like their build type, city, etc; However, after clustering, the project's main purpose is to analyze the energy consumption of each cluster. So the ...
user avatar
5 votes
1 answer
287 views

(Why) Are stepwise regression coefficients biased?

Issues with stepwise regression are known to statisticians. It yields R-squared values that are badly biased to be high. The F and chi-squared tests quoted next to each variable on the printout do ...
user avatar
  • 31.1k
0 votes
0 answers
26 views

panel data model and variable selection?

I am looking for a reference to perform variable selection with panel data. I have 9 variables and want to test which variables to keep or remove. Someone recommended step-wise analysis, but I cannot ...
user avatar
  • 1
0 votes
1 answer
63 views

Regression in R - dummy variables

Hey I want to build a model in R and one of my idependent variable is categorical (it takes 10 different values). I change the type of this variable from "char" to factor and build a model ...
user avatar
  • 117
0 votes
1 answer
76 views

Reducing a linear regression (OLS) model by dropping non-significant coefficients

Would it be proper for me to reduce a model by iterating though the coefficients and dropping the ones with high p-values and then refitting and doing this again until all coefficients are significant?...
user avatar
  • 441
0 votes
1 answer
132 views

Does the column ordering matter in the stepwise algorithms used by R?

Suppose I have a large data set with variables $x_1, x_2, \ldots, x_p$ to predict response $y$ where $p$ is very large (however $n >> p$). I would like to perform forward stepwise regression on ...
user avatar
  • 175
0 votes
0 answers
35 views

Forward selection metrics

When using forward selection for multiple linear regression, I've seen several metrics: (1) Using MSE - at each step, try adding each variable one at a time, see which variable reduces the MSE the ...
user avatar
2 votes
0 answers
35 views

Is it correct to combine any feature selection (backward/forward/stepwise) with regularization in logistic regression?

I use stepwise regression for exclude "worst" features (based on p-value) and after try to build model with L2 regularization on selected features. Emperically, this model is better that ...
user avatar
1 vote
1 answer
79 views

Reference request: accounting for AIC stepwise model selection with bootstrapped standard errors

The use of the AIC for model selection via comparison over many models is well-known and rightly maligned. But I read an interesting passage from Hastie et. al.'s classic The Elements of Statistical ...
user avatar
  • 1,033
4 votes
1 answer
151 views

On the test of the fixed effects parameters in glmer (stepwise selection)

I'm using a logistic mixed-effect model with random intercept and I want to perform a stepwise selection of the parameters of the fixed-effects. Is there any way to perform s stepwise selection using ...
user avatar
  • 135
0 votes
1 answer
145 views

How to choose first variable to do forward selection with (using p)?

I have four variables, and I'm supposed to forward select (using their p values) them at the 5% level. Normally you'd probably start with the variable that has the lowest p value, but I have some ...
user avatar
1 vote
1 answer
109 views

Different selection criterion (not AIC) for stepwise model selection in R?

I'm trying to use the step() function in R for variable selection of a linear model. My model looks like this: ...
user avatar
1 vote
1 answer
104 views

How does forward stepwise regression select variables

Suppose there are $p = 3$ variables total and suppose the forward stepwise procedure selects the third variable. The forward stepwise procedure will assign it a positive coefficient if and only if the ...
user avatar
  • 1,613
0 votes
0 answers
56 views

Stepwise AIC - on which model is the forward- and backward step performed

After I have read on different websites different claims, I hope that someone could may help me with the following question. :) I refere to stepwise AIC w.r.t regression models. In fact, I wonder on ...
user avatar
0 votes
1 answer
33 views

Can we extract the non-significant variables by using stepwise algorithm?

In my logistic regression model, I want to remove all the significant variables and keep only the non-significant ones. Because I want to have only the variables that are not distinguished between « ...
user avatar
  • 135
0 votes
0 answers
19 views

Fitting a model after step with missing values

If I initially fit a linear model using only complete cases, and I then perform stepwise regression to obtain a nested model, which no longer includes some of the variables which had missingness. ...
user avatar
  • 556
1 vote
1 answer
519 views

Multiple linear regression: p-value=0.25 pre-filter variable selection

I have used many times in a multiple logistic regression the criteria of p-value=0.25 like pre-filter variable selection using bivariate logistic regression , then I use a MANUAL stepwise (backward) ...
user avatar
1 vote
0 answers
65 views

How to find the best model when there are lots of possible interactions

I have a dataset with 6 variables such as price, country (this is a categorical variable with 7 levels), Rating_A, Rating_B, Rating_C, and Rating_D. However, if I fit the GLM like ...
user avatar
  • 11
5 votes
1 answer
111 views

Is overfitting an issue if all I care about is training error

I am working on a project where we perform non-response adjustment by weighting survey respondents by their probability of response. In order to do this, we need to estimate each respondents ...
user avatar
  • 1,408
18 votes
4 answers
2k views

Is Fig 3.6 in Elements of Statistical Learning correct?

Here is the figure from the textbook: It shows a decreasing relationship between subset size $k$ and mean squared error (MSE) of the true parameters, $\beta$ and the estimates $\hat{\beta}(k)$. ...
user avatar
0 votes
0 answers
98 views

How to report findings from regression model chosen using stepwise AIC

I want to test the effect of several explanatory variables on an independent. I am using stepwise AIC to chose the best model. Given that I have found the 'best model' how would you recommend I report ...
user avatar
0 votes
2 answers
354 views

Are repeated cross validation, forward feature selection, and LASSO compatible?

I am working on building a predictive model on medical images based on global images features extraction (texture). As I only have 51 patients, I want to build a simple model with cross-validation. ...
user avatar
  • 13
0 votes
0 answers
51 views

Stepwise Regression for VAR or VECM Models

I'm doing an analysis of integrated price series for two different types of crude oil. These crudes are priced off a differential to a main type ("marker"). I built an exogenous variable ...
user avatar
2 votes
1 answer
65 views

How to deal with predictors which are not significant, although r-squared is significant?

I did factor analysis and found three factors. To examine if which factors significantly affected a certain dependent variable, I added all three factors to a regression model. The correlation ...
user avatar
  • 21
3 votes
2 answers
47 views

R: Multiple Linear Regression - Prediction model with 4 Indenpendent Continuous Variables

We are tasked to build a linear model to predict the current of the river based on river width, river depth, distance to the ocean and bank height in order to understand the variation in stream ...
user avatar
  • 31
3 votes
2 answers
125 views

Significance of epidemiological confounders in a generalized linear model

I am identifying risk factors for children snoring among several predictors with generalized linear model. With backward selection, age and sex do not appear to be significant so I removed them from ...
user avatar
0 votes
0 answers
298 views

Backward stepwise selection stopping rule in an ordinal logistic regression context when a model's ranking ability is of importance

First off, I am aware that there are some problems with stepwise regression as for instance described here ;) I am saying this to avoid that the discussion goes in the direction of stepwise being an ...
user avatar
  • 153
2 votes
1 answer
2k views

Forward and backward stepwise regression (AIC) for negative binomial regression (with real data)

I am doing some count data analysis. The data is in this link. Column A is the count data, and other columns are the independent variables. At first I used Poisson regression to analyze it: ...
user avatar
  • 23
3 votes
1 answer
184 views

Understanding 'aggressiveness' of lasso, forward stepwise selection and best subset selection in Hastie, T., Tibshirani, R. & Tibshirani, R.J. (2017)

Hastie et al. (2017) explain how the above mentioned methods perform depending on the signal-to-noise ratio (SNR) with their varying 'aggressiveness'. Now I don't understand why the different methods ...
user avatar
  • 115
0 votes
0 answers
46 views

Best subset regression with instrumental variable

I am applying multiple regression with a data. There are 19 regressors in total and one of them is endogenous. For the endogenous variable I have identified an instrumental variable. When I apply ...
user avatar

1
2 3 4 5
7