Linked Questions
45 questions linked to/from Can a random forest be used for feature selection in multiple linear regression?
2
votes
2
answers
3k
views
How to interpret when a variable is not significant in logistic regression while having the highest variable importance in a tree-based model [duplicate]
I'm building a binary classifier with logistic regression and boosting. Just like the case that I described in the title, I am a little bit confuse on how to explain the result of those two models ...
- 147
1
vote
0
answers
65
views
can i use random forest for feature selection and then use poisson regression for model fitting? [duplicate]
Variables that are important in random forest don't necessarily have any sort of relationship with the outcome. So would it be wise to use random forest to gather the most important features and then ...
- 11
141
votes
9
answers
59k
views
Obtaining knowledge from a random forest
Random forests are considered to be black boxes, but recently I was thinking what knowledge can be obtained from a random forest?
The most obvious thing is the importance of the variables, in the ...
- 4,004
31
votes
8
answers
34k
views
When to avoid Random Forest?
Random forests are well known to perform fairly well on a variety of tasks and have been referred to as the leatherman of learning methods. Are there any types of problems or specific conditions in ...
- 575
57
votes
2
answers
38k
views
When will L1 regularization work better than L2 and vice versa?
Note: I know that L1 has feature selection property. I am trying to understand which one to choose when feature selection is completely irrelevant.
How to decide which regularization (L1 or L2) to ...
- 5,553
26
votes
2
answers
7k
views
Do all machine learning algorithms separate data linearly?
I am an enthusiast of programming and machine learning. Only a few months back I started learning about machine learning programming. Like many who don't have a quantitative science background I also ...
- 2,211
8
votes
5
answers
8k
views
When does logistic regression not work properly?
I need to find a situation in which logistic regression does not work well. Furthermore, I would like to know when a random forest might perform better than a logistic regression model.
- 131
12
votes
2
answers
9k
views
Using LASSO only for feature selection
In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization.
My question: do people normally use ...
- 121
9
votes
2
answers
592
views
Can any data be learned using polynomial logistic regression
We know that a Taylor polynomial can approximate any continuous function. As @DemetriPananos noticed, Logistic regression seeks to estimate the coefficients for a model and any cut off is imposed post ...
- 531
6
votes
2
answers
3k
views
Random forest and LASSO regression both give different variable importances
I have a dataset with 163 observations (all countries in the world with population > 1000000) and 290 variables related to their disease burden and performance. Because there are more variables than ...
- 371
4
votes
1
answer
2k
views
How To Deal With Large Numbers Of Categorical Predictors
I have three data sets that, when joined, have O(320) independent variables for a classification problem.
Principal component analysis (PCA) seems out of the question because the data is mostly ...
- 143
8
votes
1
answer
8k
views
AUC score less than 0.5 for logistic regression
I've tested out various feature selection methods, such as the F-test, Mutual Information and the Extra Tree (Extra Randomised) Forest Classifier (ETC) as well as PCA (which is technically a feature ...
- 305
2
votes
1
answer
4k
views
Should I select features before using decision tree?
Since decision tree don't use all the input features and select them in the process, is it useful to do feature selection before?
As I see it, choosing features will decrease computing time (and ...
- 83
2
votes
1
answer
765
views
Using trees after variable selection using Lasso/Random
I am new into Machine Learning so please excuse me if my question is naive.
My question is, is it possible to use trees for example rpart or ctree after variable selection procedures such as Lasso/...
- 267
4
votes
1
answer
1k
views
Best practices for feature selection?
I have datasets that range from ~2000-9000 columns of predictor variables. I'm often charged with primarily classification - but sometimes regression tasks. I know that I don't need this many ...
- 1,609