"Stepwise regression includes regression models in which the choice of predictive variables is carried out by an automatic procedure." [Wikipedia]

learn more… | top users | synonyms

0
votes
1answer
33 views

Complete separation and stepwise regression - possible in R?

I've been using stepAIC to narrow down my logistic regression model. However, I get the following warning when I run my model: glm.fit: fitted probabilities numerically 0 or 1 occurred I know this ...
1
vote
0answers
40 views

Convergence analysis for forward stagewise regression?

Forward stagewise regression is a simple model selection algorithm related to least angle regression and LASSO. (see e.g. the LARS paper) It repeats the following steps, initializing a predictor ...
0
votes
0answers
31 views

Random Forests with almost 200 predictors

I have a data set I'm playing around with that has almost 200 independent variables. What is the best way to limit this down? My response in binomial so I was thinking making I could use a GLM and ...
0
votes
0answers
29 views

How is cross-validation used for logistic regression?

I have a fundamental question about cross-validation in logistic regression. I would really appreciate some help since something is still unclear to me. My situation is the following: I split my data ...
1
vote
1answer
62 views

Is it valid to get better performance in logistic regression using only a subset of the coefficients?

I have an imbalanced data set containing 12% of the positive class 88% negative. First, I ran a logistic regression with all my coefficients and got an average accuracy of 0.91 (I know that's not ...
0
votes
1answer
47 views

Is multicollinearity an issue when doing stepwise logistic regression using AIC and BIC?

As far as I understood, it should not be a problem as long as I don't have perfect multicollinearity since I don't mind if the standard errors get inflated. However, what about using the ...
4
votes
1answer
126 views

Linear regression - iterative approach

I have a single output variable $y$ and a number of inputs $x_1$, $x_2$, etc. These are time series. Each $x_i$ explains the changes in $y$ in specific circumstances, and the goal is to have a linear ...
3
votes
1answer
41 views

How can I prove that the f-statistic does not follow an F distribution in the context of step-wise regression?

There is a good number of threads about the deficiencies of step-wise regression, and particularly on the shortcomings of the partial F test as a tool for step selection. However I find it difficult ...
0
votes
0answers
50 views

Interpreting AIC forward stepwise function in R

My homework asks me to: "Try a forward stepwise procedure with entry probability of 0.20. Then describe the model that is arrived at and whether it might be preferred." I used the forward step ...
0
votes
1answer
42 views

How to deal with categorical variable - location- with more than 60 levels

I am new to statistics and to categorical variables. I need to predict the cost based on several variables and it happened that all of my variables are categorical. I tried doing a linear regression ...
0
votes
0answers
73 views

Stepwise regression and variable selection with categorical variables in R

I am new with statistics and especially stepwise regression with categorical variables. I have 4 categorical variables, each with a different levels (5 levels, 12 levels, 7 levels, and 78 levels). I ...
2
votes
1answer
92 views

Recommend a method for variable selection (other than classification tree or random forest)?

Just wonder if you could recommend a few methods (other than tree-based methods) to analyze a dataset in which n= 350 and p = 35. The goal is not so much about prediction, but to find/select ...
0
votes
0answers
47 views

How to do stepwise regression forward correctly?

My understanding is that you add only one single variable at a time based on various model fit or statistical criterion. Someone advances that there is merit to running a stepwise regression by ...
15
votes
2answers
802 views

Why are p-values misleading after performing a stepwise selection?

Let's consider for example a linear regression model. I heard that, in data mining, after performing a stepwise selection based on the AIC criterion, it is misleading to look at the p-values to test ...
0
votes
0answers
53 views

Choose model by BIC in a stepwise algorithm after choosing model from glmnet

I have data where number of observation n is smaller than number of variables p. The answer variable is binary. For example: ...
3
votes
1answer
126 views

What does it mean that stepwise, backward and forward selection methods are “path dependent”?

In many papers I read that stepwise, backward and forward selection methods are "path dependent". What does it mean? Could anyone give me some practical example to understand the underlying concept? ...
0
votes
1answer
45 views

stepwise, forward and backward selection when the regressors are too much correlated

Why automated variable selection methods like stepwise regression, backward elimination and forward stepwise regression are not suitable when the regressors suffer from multicollinearity? Could anyone ...
1
vote
0answers
36 views

How does SAS's stepwise logistic work?

Not being accustomed to reading documentation as a sequence of tables, I'd like to know if someone could kindly explain how SAS's Proc Logistic invocation with selection=backwards works. ...
1
vote
1answer
164 views

How to decide which interaction terms to include in a multiple regression model?

I am trying to build a multiple regression model using R. I have a number of predictor variables. I have some basic domain knowledge for which I am trying to build the model. To start with, I included ...
0
votes
0answers
38 views

R stepAIC multiple datasets

I have three datasets of similar(ish) sizes (268, 271 and 262), and a model I am trying to develop for describing a response variable within the data. I'm trying to use the ...
1
vote
2answers
67 views

Using correlation to eliminate predictors? [duplicate]

I have 1 dependent variable and 33 independent variables (continuous, categorical & dichotomous). Correlation analyses (2-tailed) show that the DV is only correlated to 7 of the IVs although most ...
0
votes
1answer
37 views

Can a variable become statistically significant after the addition of another variable? [duplicate]

I am doing forward stepwise logistic regression. I have heard that its common for a previously statistically significant variable to become not statistically significant when one or more variables are ...
0
votes
0answers
26 views

Is it possible to use stepwise linear regression to find the parameters that explain the most variation in the output of a nonlinear system?

I have a mathematical model of a system. We have to use a simulation software to check the response of the system to specific input signals or change of model parameters. The relationship between some ...
0
votes
1answer
8 views

Detecting outlying distributions of ratio data

I have a dataset consisting of hundreds of repeat observations on thousands of agents. Each observation is a ratio between two distance measures, A and B, where A is always larger than B. Thus, my ...
1
vote
1answer
335 views

Combining principal component regression and stepwise regression

I want to use a combination of principal component analysis (PCA) and stepwise regression to develop a predictor model. I have 5 independent variables (which are correlated among each other to ...
0
votes
0answers
46 views

Variable reduction techniques

I am researching variable reduction techniques for time series data. Atm I came up with expert judgement, Stepwise Regression (Forward), Stepwise Regression (Backward) and Granger Causality. Any ...
2
votes
0answers
104 views

R: Dynamic Regression with ARIMA model using xreg, make use of step function?

This might fit better here than on stackoverflow, I guess. I was trying to build a dynamic regression model with the dynlm package, but it did not work out. After reading this by Hyndman, I now ...
0
votes
1answer
233 views

Significant predictors of mpg in mtcars dataset in R

Which variables are significant predictors of mpg in mtcars dataset. When I perform following regression: ...
2
votes
2answers
324 views

What is the difference between VIF and stepwise regression?

What is the difference between the variance inflation factor (VIF) and stepwise regression as both help in detecting multicollinearity? What variables are different while running both techniques?
0
votes
0answers
73 views

Do I drop insignificant parameters from a model? Should I use stepwise regression?

I'm working on a project where we have a number of factors we believe might have a role on a survey result. It's my job to figure out if this is true. My boss suggested just doing correlations on each ...
1
vote
0answers
120 views

R: Why does step function of a Linear Modegives different AIC/BIC than AIC function?

I don't understand what I make or think wrong, but if one tries to evaluate the linear model of the data (which you can find in R in the Package AIC(stats)), then ...
0
votes
0answers
244 views

How to use residual analysis to remove the effect of confounding variables in a model in R

I want to find which soil variables better explain plant productivity, using a database that contains information for about 100 forests plots across Europe. These plots have only one species per plot, ...
1
vote
1answer
212 views

Stepwise logistic regression

I am working with a dataset of 1000 individuals, 200 of which are disease positive. I have run a logistic regression with 25 predictors to identify overall which variables are significantly ...
2
votes
2answers
179 views

What is the best lag length for auto correlation?

I am doing a monthly rainfall forecasting model. I have monthly data from 1998 to 2012. I found in previous research that they have used partial autocorrelations and stepwise regression as an input ...
1
vote
1answer
112 views

Validity of stepwise regression in DistLM

I have a set of nutrient fluxes data and I would like to know which environmental drivers explains the fluxes. I used DistLM and the marginal test showed that none ...
0
votes
0answers
20 views

Is there some analysis of block-greedy algorithms for feature selection or sparse approximation?

I consider the problem of sparse approximation, where one has a signal $\vec y = \sum_j \theta_j \cdot \phi_j(\vec x)$ using stepwise regression. One can use a greedy algorithm to solve it, e.g. ...
1
vote
1answer
138 views

GAMLSS: model with interaction terms failed

I use gamlss method from library(gamlss) on my full models with interaction terms and try to reduce them with stepGAIC. There are 3 things I want to ask. Do I have to specify a link for the model? ...
3
votes
1answer
5k views

Stepwise Model Selection in Logistic Regression in R

I'm implementing a logistic regression model in R and I have 80 variables to chose from. I need to automatize the process of variable selection of the model so I'm using the step function. I've no ...
3
votes
2answers
567 views

Should I remove non-significant variables from my regression model

I have run a multiple linear regression using stepwise regression to select the best model, however the best model returned has a non-significant variable. When I remove this the AIC value goes up ...
1
vote
1answer
569 views

dummy variables, interaction with continuous variable, and variable selection

I want to predict shop sales from a set of independent variables which consists of shop attributes like floor space, no. of stuff of a specific store (continuous variables) and also location of the ...
1
vote
1answer
341 views

step {stats} is too slow. Are there multicore solutions?

I am finding that trying to do a stepwise logistic regression is far too slow on my data set (6 hours). Is anyone aware of any faster solutions out there? Perhaps one that takes advantage of the ...
0
votes
1answer
83 views

Selection of regressors into a regression model

Why is it that backward selection/elimination as compared to forward selection of regressors, is often less adversely affected by the correlative structure of regressors?
0
votes
1answer
246 views

Linear regresson lm or stepwise regression here using R?

It is a basic question but I could not find clear answer on my reading. I am trying to find independent predictors of Infant.Mortality in data frame 'swiss' in R. ...
1
vote
3answers
221 views

What is the basis of setting critical p-value value in stepwise regression?

In statistical software like MINITAB and SAS, the default alpha value (critical p-value) is set as 0.15. I would like to know (1) if there is any statistic basis to set it as 0.15 and (2) if this is a ...
2
votes
0answers
1k views

covariate selection for a cox model by Lasso using glmnet

I would like to use model selection through shrinkage (Lasso) using glmnet. So far I did the following: ...
1
vote
1answer
588 views

(Automated) feature selection in multiple regression with categorical variables

I need a general guide on what are the appropriate approaches to automated feature selection in multiple regression with categorical variables. In my case, I have several numeric and categorical ...
1
vote
2answers
1k views

Backward selection for Cox model using R

I want to perform an exploratory Cox regression analysis of medical data using R. I am practicing using the pbc data from the survival function. Would you recommend performing a backward selection ...
1
vote
2answers
391 views

Why is Lasso regression for high dimensional data better than Stepwise AIC?

I know Lasso eventually set some parameters to zero, acting like variable selection. I also read from paper talking about automated variable selection method like Stepwise AIC can be troublesome. So ...
1
vote
1answer
714 views

Highly correlated predictors in backward stepwise regression?

I know that it's not right to enter variables having multicollinearity (high correlation) into a regression analysis. But if I'm using backward stepwise regression could I add all the highly ...
0
votes
1answer
141 views

How to run main effects and interactions in a stepwise regression?

I am using multiple regression with the backward elimination method. I have one control variable (social desirable responding) and four predictor variables (gender and three self-esteem constructs). ...