"Stepwise regression includes regression models in which the choice of predictive variables is carried out by an automatic procedure." [Wikipedia]

learn more… | top users | synonyms

0
votes
0answers
20 views

How do you extract confidence intervals and OR out of the step() function in R?

I've been wondering something for a while. If you run a simple regression model in R and then perform a step-wise selection (it doesn't have to be the way I typed the code below), how do you extract ...
0
votes
0answers
21 views

using stepAIC of MASS package to select variables with a significance level of 5% in R project

First of all, sorry i am new about this and any helps are really welcome. I am reading a reaserch paper where the authors report: Stepwise forward regression (Zar 1996) was used to select the most ...
0
votes
0answers
24 views

R forward stepwise regression [migrated]

In R stepwise forward regression, I specify a minimal model and a set of variables to add (or not to add): ...
0
votes
0answers
4 views

Error using the step-function with glmmML [migrated]

When I tried to use the step function I receive this error: "Error in if (all(is.finite(c(n0, nnew))) && nnew != n0) stop("number of rows in use has changed: remove missing values?") : ...
0
votes
0answers
35 views

Stepwise meta-regression with R (metafor)

I am using the "metafor" package to do a multivariate meta-regression in "R". I have 6 predictors and I am able to run the full model (all the predictors simultaneously in the model) just fine. ...
0
votes
1answer
23 views

R not testing certain variables in forward stepwise regression?

FullModel<- (lm(Fubar~.-Foo-Bar,data=BarFoo)) NullModel<-(lm(Fubar~1)) step(NullModel,scope=formula(FullModel),direction="forward",k=log(nrow(BarFoo))) When ...
0
votes
1answer
79 views

Superiority of LASSO over forward selection/backward elimination in terms of the cross validation prediction error of the model

I obtained three reduced models from a original full model using forward selection backward elimination L1 penalization technique (LASSO) For the models obtained using forward selection/backward ...
1
vote
1answer
55 views

Linear model predictor selection. Which method to use ?

From what I understand, there are 3 main types of predictor selection method for linear models, namely, 1 Subset Selection, 2 Shrinkage and 3 Dimension Reduction. The subset selection includes the ...
4
votes
3answers
327 views

Variable is significant through stepwise regression but not in final model's summary; which should I report?

I used generalized linear mixed models (with the glmmADMB package) to identify environmental factors related to parasite abundance in rodents. I used stepwise ...
1
vote
0answers
162 views

Backward stepwise regression with cross validation in R

I would like to do model selection using backward stepwise procedure and cross validation. https://www.otexts.org/fpp/5/3 I have used stepAIC in ...
1
vote
1answer
50 views

AIC values and their use in stepwise model selection for a simple linear regression

The Wikipedia article for AIC says the following (emphasis added): As an example, suppose that there were three models in the candidate set, with AIC values 100, 102, and 110. Then the second ...
4
votes
1answer
67 views

Variable selection for regression - the subselect package

No regular here will be unaware of the perils of using stepwise and similar automatic methods for variable selection in regression analysis. But preferred alternatives, such as the lasso or ...
0
votes
1answer
58 views

Interaction effect in stepwise regression

I am trying to creat a multiple regression model with a forward stepwise procedure. Predictors are air temperature, soil temperature, PAR and snow depth. I also want to see if there are some ...
2
votes
1answer
108 views

Stepwise regression, moderation effects, main effects

I have a simple model: $A$ is hypothesized to be a predictor / regressor / explanatory / input variable $B$ is hypothesized to be the response / regressand / explained / outcome variable So, the ...
0
votes
0answers
88 views

Model estimation procedure using backward elimination

I have run a multiple linear model using Minitab. The result showed that all variables are not significant. So, I use backward elimination. Lastly, I found that ...
1
vote
1answer
54 views

Numeric example of data for special case of stepwise linear regression

Stepwise Regression works as follows if I'm correct: fit the initial model add the variable which has its f-stat larger than a in-threshold and repeat step 2. if there are no candidates to enter - ...
2
votes
0answers
62 views

Does full subset selection regression model building suffer from the same handicaps as stepwise regression?

Let's assume $p$ potential predictor variables $X_1,...,X_p$ and a single dependent variable $Y$. Now I evaluate the performance of all possible linear models considering all possible combinations of ...
0
votes
0answers
112 views

How to build the best regression model using stepwise regression?

I have a data set that contains rental cost (Dependent variable) and I have another data sets that are potential predictors like city, area, No. of rooms etc. (Independent variables). I have ...
1
vote
1answer
218 views

Generalized linear mixed models: model selection

This question/topic came up in a discussion with a colleague and I was looking for some opinions on this: I am modeling some data using a random effects logistic regression, more precisely a random ...
1
vote
2answers
159 views

Logistic regression, SPSS ignores my reference category and assumes another one

I am modelling logistic regressions in SPSS, the same model for different countries (well, with slight differences in the independent variables set due to collinearity diagnosis and stepwise results). ...
2
votes
2answers
60 views

Does the number of IV after stepwise model selection depend on the amount of data?

I have 3 different DV that I try to model with 3 distinct models (linear mixed models) using the same set of IV. I found that the DV that I have the least amount of data for also has the lowest number ...
4
votes
1answer
135 views

Alternatives to stepwise discriminant analysis for feature selection on hyperspectral data

I am new to R and to hyperspectral data analysis. However, in my research, I have found that many warn against using Stepwise discriminant analysis (using Wilk's Lambda or Mahalanobis distance) for ...
5
votes
2answers
800 views

How does “stepwise regression” work?

I used the following R code to fit a probit model: ...
1
vote
1answer
162 views

Implement Forward, Backward, Step and LASSO in VB .NET

My client wants me to implement Variable selection methods i.e. Forward, Backward, Step and LASSO in VB .Net platform including p-value and AIC. I have no idea about the steps involved to calculate ...
1
vote
1answer
140 views

Variable selection for linear regression using robust or least squares estimation

I have a data set consisting of one continuous response variable and about 70 predictors. Using this data, I want to construct a linear regression model. However, I don't know what predictors are ...
6
votes
1answer
352 views

What does an infinite AIC mean and what can be done about it?

I have a question about performing stepwise regression. I realize that there are issues with using stepwise methods, but I have about 30 or so predictors and have constructed an ...
4
votes
1answer
223 views

Interpretation of insignificant predictors in logistic regression model

First I should explain what I did, and it might not be right. I have a variable that represents a test outcome, it might be positive or negative. I have a set of observations of one important ...
1
vote
2answers
667 views

Interpretation of coefficients in multiple regression without intercept

I am trying to interpret the SPSS output from a multiple hierarchical regression where the intercept has been eliminated because it is not significant. I have read previous discussions about ...
1
vote
1answer
139 views

Stepwise versus L2 regularized logistic regression: dataset-specific performance

I have two data sets from different collections. The second data set is smaller. They were both analyzed with the same methods in order to derive feature sets of 10-30 features each. Each feature set ...
3
votes
1answer
193 views

The dangers of stepwise variable selection in regression

This paper discusses some of the dangers of using stepwise variable selection procedures: http://www.auburn.edu/~tds0009/Articles/Whittingham%20et%20al.%202006.pdf I'm struggling to understand ...
0
votes
1answer
438 views

Stepwise introduction of predictors to mixed-effects models

As the title says, what I'd like to do is stepwise introduction of predictor variables to a mixed-effects model. I'm going to first say what I'd be doing if it were stepwise linear regression, just to ...
1
vote
1answer
60 views

Estimation Technique

My panel regression model is as follows: $$Y_{it}= PS_{it}+PF_{it}+EF_{it}+ e_{it}$$ where $i$ : country $t$ : year $Y_{it}$ : GDP per capita $PS_{it}$ : Political stability $PF_{it}$ : ...
2
votes
1answer
163 views

How to perform step() when n < p in R?

I am trying to perform stepwise regression for variable selection in R. In matlab, the stepwisefit function is able to work in ...
0
votes
3answers
615 views

Is it possible to have a variable significant in multiple regression but not significant in stepwise regression?

I have run a stepwise regression and found that some of the selected variables are not significant yet in a multiple regression with all variables included in the model those variables were ...
0
votes
1answer
378 views

fastbw with rule=“p” in R's rms package: why do results depend on number of covariates?

I've been trying to use the fastbw function from the rms package in R to perform logistic regression with backward selection, with p-values as exclusion criterion (I am well aware of the arguments ...
0
votes
1answer
180 views

Software implementation of stepwise regression after multiple imputation

Simple question, does anyone know of a package (R preferred, but I'll take anything, SAS, Stata, SPSS) which implements stepwise regression of multiply imputed datasets. I've read that it's possible ...
3
votes
0answers
134 views

Fast algorithm for variable selection

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive ...
10
votes
1answer
1k views

What is the difference between AIC() and extractAIC() in R?

The R documentation for either does not shed much light. All that I can get from this link is that using either one should be fine. What I do not get is why they are not equal. Fact: The stepwise ...
2
votes
1answer
270 views

Specifying add and drop thresholds for stepwise regression in R

I am running a stepwise regression using the F test as the criterion. Is there a way to explicitly set the add and drop thresholds (alpha levels) in R? The documentation does not make it clear.
2
votes
2answers
418 views

How to conduct predictor selection in a generalized linear mixed model?

I have 18 predictors in a binary generalized linear mixed model (repeated measurements, over a 1000 subjects). I would like to trim the model a bit and remove some noise and useless predictors. ...
4
votes
1answer
579 views

Stepwise regression vs. elastic net

I understand that Stepwise regression analysis has lots of limitations, including the assumption that the predictors are not highly correlated with each other. In fact, this limitation was the most ...
2
votes
2answers
289 views

R-code question: model selection based on individual significance in regression?

I'm trying to generate an R function that keeps relevant variables based on their absolute t-value (or p, whichever is easier in code). Basically what I want is to run one regression (1), retain all ...
1
vote
0answers
261 views

Stepwise Regression Models in JMP

In JMP, I am building a regression model by using "Analyze"->"Fit Model" and choosing "Stepwise" for the personality. Once I click "Run" in the "Model Specifications" window, I get the "Fit Stepwise" ...
17
votes
5answers
2k views

Detecting significant predictors out of 300 independent variables

In a dataset of two non-overlapping populations (patients & healthy, total $n=60$) I would like to find (out of $300$ independent variables) significant predictors for a continuous dependent ...
5
votes
3answers
785 views

Does a stepwise approach produce the highest $R^2$ model?

When using the forward stepwise approach to select variables, is the end model guaranteed to have the highest possible $R^2$? Said another way, does the stepwise approach guarantee a global optimum or ...
7
votes
4answers
2k views

Is there a way to use cross validation to do variable/feature selection in R?

I have a data set with about 70 variables that I'd like to cut down. What I'm looking to do is use CV to find most useful variables in the following fashion. 1) Randomly select say 20 variables. ...
35
votes
4answers
8k views

Algorithms for automatic model selection

I would like to implement an algorithm for automatic model selection. I am thinking of doing stepwise regression but anything will do (it has to be based on linear regressions though). My problem ...
3
votes
2answers
3k views

Assessing the effect of adding a variable using stepwise forward logistic regression using Stata?

I'd really appreciate help using Stata to perform a manual stepwise forward logistic regression. I have 37 biologically plausible, statistically significant categorical variables linked to disease ...
40
votes
5answers
4k views

What are modern, easily used alternatives to stepwise regression?

I have a dataset with around 30 independent variables and would like to construct a GLM to explore the relationship between them and the dependent variable. I am aware that the method I was taught ...
2
votes
0answers
2k views

Forcing variable selection to keep certain predictor in R

I am looking for a variable selection technique in R to reduce the number of my regression predictors, where I can force the method to keep a specific variable within the model. Here is a toy example ...