Linked Questions

195
votes
8answers
87k views

Algorithms for automatic model selection

I would like to implement an algorithm for automatic model selection. I am thinking of doing stepwise regression but anything will do (it has to be based on linear regressions though). My problem ...
14
votes
2answers
83k views

Stepwise regression in R - How does it work?

I am trying to understand the basic difference between stepwise and backward regression in R using the step function. For stepwise regression I used the following command ...
6
votes
3answers
10k views

Should I remove non-significant variables from my regression model

I have run a multiple linear regression using stepwise regression to select the best model, however the best model returned has a non-significant variable. When I remove this the AIC value goes up ...
4
votes
1answer
6k views

Forward or backward sequential feature selection?

I was trying to carry out feature selection on a dataset using sequential feature selection. The dataset contains more than 5000 observations (rows) and 22 features (columns). Now I see that there are ...
5
votes
1answer
4k views

Combining principal component regression and stepwise regression

I want to use a combination of principal component analysis (PCA) and stepwise regression to develop a predictor model. I have 5 independent variables (which are correlated among each other to ...
2
votes
1answer
784 views

Investigating interaction

Please I need to check for interaction before building an explanatory model (logistic regression). I have 16 interaction terms in total. Please how what is the best way to go about it. Will I need to ...
2
votes
2answers
113 views

F-statistics based method out of fashion?

I'm reading Elements of Statistical Learning and come across this paragraph right before section 3.3.3: Other more traditional packages base the selection on F -statistics, adding “significant” ...
2
votes
2answers
483 views

Feature Selection in unbalanced data

I was always taught 3 things: Training algorithms (rf, trees, etc) don't perform well with unbalanced data. I should balance data only after performing feature selection (mainly to keep variables ...
1
vote
1answer
54 views

Should I report the pseudo $R^2$ value for full or final logistic regression model after removing NA's & running stepwise selection?

I'm working with a logistic regression model in r. model <- glm(response~., family="binomial", data) and I'm using ...
0
votes
0answers
69 views

Eliminate Qualitative Predictors (working in R)

So I'm a totally newbie to the field of statistics. I'm working on a project where I have a quantitative dependent variable that I'm supposed to predict using a mixture between quantitive and ...
0
votes
1answer
37 views

Understanding regression modelling: 3 factors, 3 continuous predictors

I am a bit confused about how regression modelling works. I have a response $y$, 3 continuous predictors, and 3 factors. I don't have anything else available. I fit the model ...
0
votes
2answers
22 views

The sufficiency of univariate selection

Why is feature selection such a common topic, when one could simply use SelectKBest to find the optimal set of features? The only trouble would be to find the best ...
0
votes
0answers
33 views

My predictors are all categorical variables but the dependent is numerical, how to eliminate dummies?

My predictors are all categorical values but the dependent is numerical. How can I eliminate dummies if I use a linear regression model? The values are tough to solve with backward elimination; ...