Linked Questions

0
votes
0answers
172 views

Why Stepwise regression lead to overfitting? [duplicate]

I run a rolling backward stepwise factor selection within each regression window with a Matrix of regressors X(137x481) and a vector Y(1x137). As you can see, the number of regressors is way higher ...
200
votes
8answers
90k views

Algorithms for automatic model selection

I would like to implement an algorithm for automatic model selection. I am thinking of doing stepwise regression but anything will do (it has to be based on linear regressions though). My problem ...
9
votes
1answer
8k views

Why lasso for feature selection?

Suppose I have a high-dimensional dataset and want to perform feature selection. One way is to train a model capable of identifying the most important features in this dataset and use this to throw ...
4
votes
3answers
373 views

What variables need to be controlled for in regression?

There are numerous discussions on this site concerning how to control for certain variables in regression analysis. How exactly does one “control for other variables”? How do you "control" ...
0
votes
0answers
2k views

What is the difference between Stepwise regression and Lasso regression in terms of variable selection?

What is the difference between Stepwise regression and Lasso regression in terms of variable selection? Is the difference just the way in which the variables are selected or is there any significant ...
2
votes
1answer
483 views

Model complexity in cross-validated stepwise regression

Suppose that we perform forward stepwise regression and use cross-validation to choose the best model size. Using the full data set to choose the sequence of models is the WRONG way to do cross-...
1
vote
2answers
296 views

How to interpret the multiple regression results?

For selection of independent variables in multiple regression analysis, first have calculated correlation coefficient between dependent (y) and independent variables (x1, x2, x3, x4 & x5) and ...
1
vote
0answers
239 views

Why avoid stepwise regression? [duplicate]

I have been using model averaging and model selection bases on AIC and BIC for a while. I have recently discover the stepwise regression technique and I found a lots of people critize this methods. ...
1
vote
1answer
58 views

In stepwise regression, how to interpret non-significant variables? [duplicate]

I have more than 15 IVs such as age, gender, education, first language, technology proficiency, health condition, etc, and one of my DVs is health literacy level, which is measured through a standard ...
0
votes
2answers
124 views

How to do model selection with more regressors than observations?

Consider a standard ordinary linear regression model with $n$ observations and $K>n$ possible regressors (i.e., more regressors than observations). You know the model is linear but you don't know ...
2
votes
1answer
108 views

GLM Categorical IV Predictor vs Group by Analysis

I am modeling a continuous dependent variable with a couple of covariates (known a priori) and a variable of interest. I ran into an issue of interpretation which I'd like to clear up. When I ...
2
votes
2answers
60 views

Interpretation of p-values in multiple regression output

I have data from a survey where I collected demographic information and quiz scores from college students. I ran backward model selection to determine which of my variables affect knowledge. My best ...