# Tag Info

### Why do we use Linear Models when tree based models often work better than linear models?

Many excellent answers already. I would add a few more aspects. You do not say what you mean by "outperform", as in "tree models often outperform linear models". You presumably ...
• 126k

### Why do we use Linear Models when tree based models often work better than linear models?

For most problems it is easy to beat a tree model with a regression model. That's because tree models allow for all possible interactions among predictors and these are seldom needed. Interactions ...
• 94.7k

### Why do we use Linear Models when tree based models often work better than linear models?

Linear regression is also existent outside machine learning where it requires much less data. Why does Machine Learning need a lot of data while one can do statistical inference with a small set of ...
• 81.3k

### Why do we use Linear Models when tree based models often work better than linear models?

In addition to the excellent answers given already, a regression model yields parameter estimates, which tree models do not. If all you are interested in is prediction (which, as I understand it, is ...
• 124k

### Why do we use Linear Models when tree based models often work better than linear models?

I share your observation that on well-behaved tabular data (quite many rows, not too many columns), a well-built boosted trees model usually beats a random forest, and a random forest usually beats a ...
• 11.9k

### Is ROC curve unique?

Maybe I don't understand your notation, but I feel you are mixing up true probabilities, model outputs (which might be probability estimates) and events. given an event E, model output $f(\theta)$ (...
• 7,075
Accepted

### Can a ML classifier's prediction be understood as a probability?

That would be desirable, but it is not guaranteed to make as much sense as we might like. First, you could make an argument that any predicted $p(\mathcal C_k|\mathbf x_i)\in[0,1]$ is a probability in ...
• 64.3k
Accepted

### Regression model with multiple rows per user to predict death

Since the response is mortality, the most appropriate method is survival analysis that explains time to death, a strictly positive value. It appears that the difference between the first ...
• 1,774

### What is the best way to meaningfully compare the feature importances between parametric (Logit) and non-parametric models (RFs/XGB)?

This is tricky, because it depends in part on (i) what you mean by importance, and (ii) what you want out of such a comparison. To the first point, there are many ways to convert the somewhat vague ...
• 18.8k
Accepted

### How to interpret the results of a classifier when train/test method gives much better results than cross validated one?

What does these varying scores represent, particularly the low scores of cross validation? Together, they represent the fact that error estimtes based on a small number of tested cases are highly ...
Accepted

### How to interpret pairplots()

A pair plot is really just a scatter plot between all combinations of two variables in your dataset. This means you can essentially ignore one of the two triangles (the lower half is the same as the ...
• 5,133
Accepted

### What is happening behind the scenes when we use CalibratedClassifierCV without prefit?

When we use cv=prefit, we will split the data into train, test and calibration sets, then fit a model using train sets, calibrate with calibration set and later use the calibrated model with the test ...
• 4,723

### meaning of drop in OneHotEncoder

Implementation-wise, the drop keyword results in one category per column being dropped. Thus, out of ['Female', 'Male'], ...
• 1,833

### How reliable is train_test_split? Is there a way to optimize it?

The aim of a train/test split is to estimate true model performance. To do this in an unbiased way, the test data needs to be independent of the training data. If rows are independent, then a random ...
• 11.9k

### Scikit-learn : MultiOutputRegressor

They are completely different and unrelated things. MLPRegressor is a neural network model used for a regression problem. There are one or more target variables to ...
• 139k

### How to evaluate the results of a multilabel classifier using the predicted probabilities?

Good for you to look at the predicted probability values themselves. They contain useful information yet often get forgotten in favor of predicting the category with the highest probability, which can ...
• 64.3k

### AUC measure for Local outlier detection in python?

The cleanest way is to apply the logistic or sigmoid function; it works in the majority of the situation, including those when you don't have access to all the scores at once. ...
• 31

### Sklearn's LogisticRegression C hyperparameter issue

There are two components in the mentioned cost function: error and regularization. When you want to change the weight of one of the components by C times you can do one of the following: multiply ...
Accepted

### Am I finding redundant columns in my data using Factor Analysis

Your approach is correct, but it's important to consider the size of your dataset. The results and interpretations you obtain from your data can be greatly influenced by the number of samples you have....
• 304

### Hierarchical clustering of a distance matrix with element weights

It sounds like this is an issue of geospatial projection. You’ll need to either transform your lat/lon into Cartesian coordinates, or modify your distance function to account for the areal distortion. ...
• 291
Accepted

### Implementations of Lasso in Python and R?

LASSO in Python With Python, in my experience, the most common implementation of LASSO (Least Absolute Shrinkage and Selection Operator) is provided by the ...
• 62.8k

### Scikitlearn: Why are hyperplane coefficients not available if kernel is not linear

Your intuition is on the right track. The usage of kernels is what makes SVMs so nice/convenient, since (very informally) it gives us the ability to build a classifier in some higher-dimensional space ...
• 1,833

### scikit learn: add lasso or ridge penalty only on subset of parameters

It is possible for ridge regression but not lasso as of scikit-learn 1.3.0. Compare the docs on the penalty weight alpha in ...

### Representing Nested Models as an SKLearn Pipeline

Here is a hack to achieve what I am looking for. Specifically, add transform to the regression estimator directly. It is relatively lightweight but somewhat ...
• 31

### Is my regularized logistic regression model overfit?

The sample size is not large enough to be able to reliably choose the shrinkage coefficient (penalty; regularization). The sample is barely large enough to estimate the overall prevalence of the ...
• 94.7k

### Is Linear Regression a good algorithm or even applicable with the distribution shown in the scatter plot I have shared in this question?

First, the linearity of linear regression is linearity in the coefficients, not in the predictors themselves or transformations of them. There's no need to restrict yourself to a single transformation ...
• 94.5k

### Finding the corners of noisy polygons

The following (using Mathematica) does not do what one would do "by eye" but that's because the data points don't fall perfectly on a desired number of line segments. This uses Mathematica's ...
• 3,940
1 vote

### Why do we use Linear Models when tree based models often work better than linear models?

You can view xgboost as a stepwise linear regression adding nonlinear inputs. So it's more a question of how complicated you want to make the model
• 7,075
1 vote

### Scikitlearn: Why are hyperplane coefficients not available if kernel is not linear

What you're missing here is that the representation of kernels as the sums of nonlinear features is not known. What's known is that these kernel are likely to be representable as those new features, ...
• 61.7k
1 vote
Accepted

### Why does the mean AURPC go down the more examples one uses?

I think the answer has to do with the probability of drawing a sample from x1=np.random.beta(a_neg, b_neg, n) being greater than a sample from ...
• 26

Only top scored, non community-wiki answers of a minimum length are eligible