Linked Questions

2 votes
2 answers

How to interpret when a variable is not significant in logistic regression while having the highest variable importance in a tree-based model [duplicate]

I'm building a binary classifier with logistic regression and boosting. Just like the case that I described in the title, I am a little bit confuse on how to explain the result of those two models ...
Eric He's user avatar
  • 147
1 vote
0 answers

can i use random forest for feature selection and then use poisson regression for model fitting? [duplicate]

Variables that are important in random forest don't necessarily have any sort of relationship with the outcome. So would it be wise to use random forest to gather the most important features and then ...
Trupti J's user avatar
141 votes
9 answers

Obtaining knowledge from a random forest

Random forests are considered to be black boxes, but recently I was thinking what knowledge can be obtained from a random forest? The most obvious thing is the importance of the variables, in the ...
Tomek Tarczynski's user avatar
32 votes
8 answers

When to avoid Random Forest?

Random forests are well known to perform fairly well on a variety of tasks and have been referred to as the leatherman of learning methods. Are there any types of problems or specific conditions in ...
tSchema's user avatar
  • 585
57 votes
2 answers

When will L1 regularization work better than L2 and vice versa?

Note: I know that L1 has feature selection property. I am trying to understand which one to choose when feature selection is completely irrelevant. How to decide which regularization (L1 or L2) to ...
GeorgeOfTheRF's user avatar
27 votes
2 answers

Do all machine learning algorithms separate data linearly?

I am an enthusiast of programming and machine learning. Only a few months back I started learning about machine learning programming. Like many who don't have a quantitative science background I also ...
Eka's user avatar
  • 2,281
12 votes
3 answers

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
Ryan's user avatar
  • 121
8 votes
5 answers

When does logistic regression not work properly?

I need to find a situation in which logistic regression does not work well. Furthermore, I would like to know when a random forest might perform better than a logistic regression model.
Seçil Gülbudak's user avatar
9 votes
2 answers

Can any data be learned using polynomial logistic regression

We know that a Taylor polynomial can approximate any continuous function. As @DemetriPananos noticed, Logistic regression seeks to estimate the coefficients for a model and any cut off is imposed post ...
mathgeek's user avatar
  • 551
6 votes
2 answers

Random forest and LASSO regression both give different variable importances

I have a dataset with 163 observations (all countries in the world with population > 1000000) and 290 variables related to their disease burden and performance. Because there are more variables than ...
user3387899's user avatar
4 votes
1 answer

How To Deal With Large Numbers Of Categorical Predictors

I have three data sets that, when joined, have O(320) independent variables for a classification problem. Principal component analysis (PCA) seems out of the question because the data is mostly ...
duffymo's user avatar
  • 143
7 votes
2 answers

How to fit logistic regression to circular data?

I've made a script that can do normal logistic regression with sigmoid(linear model). However, I have data that has a circular decision boundary and looks like this. My question is how I can modify ...
Eirik's user avatar
  • 71
8 votes
1 answer

AUC score less than 0.5 for logistic regression

I've tested out various feature selection methods, such as the F-test, Mutual Information and the Extra Tree (Extra Randomised) Forest Classifier (ETC) as well as PCA (which is technically a feature ...
Jayjay95's user avatar
  • 305
2 votes
1 answer

Should I select features before using decision tree?

Since decision tree don't use all the input features and select them in the process, is it useful to do feature selection before? As I see it, choosing features will decrease computing time (and ...
CoMartel's user avatar
2 votes
1 answer

Using trees after variable selection using Lasso/Random

I am new into Machine Learning so please excuse me if my question is naive. My question is, is it possible to use trees for example rpart or ctree after variable selection procedures such as Lasso/...
user3571389's user avatar

15 30 50 per page