# Tag Info

### Why using StandardScaler from sklearn before LogisticRegression increase avg cross_val_score? Standarization should only help in faster convergece

By default, sklearn logistic regression is penalized. From the documentation: penalty{‘l1’, ‘l2’, ‘elasticnet’, ‘none’}, default=’l2’ Specify the norm of the penalty: ...
Accepted

### How to find nearest neighbors using cosine similarity for all items from a large embeddings matrix?

Actually, we can use cosine similarity in knn via sklearn. The source code is here. This works for me: ...
Accepted

### Why does univariate Mahalanobis distance not match z-score?

Your intuition about the Mahalanobis distance is correct. However, the EllipticEnvelope algorithm computes robust estimates of the location and covariance matrix which don't match the raw estimates. (...

### Suitable approach to cluster histogram-like dataset using HDBSCAN implementation in python

Don't ask about the algorithm: focus on solving your problem. The peak-finding solution I posted at https://stats.stackexchange.com/a/428083/919 will help you analyze the situation and decide how many ...

### Stratification of the continuous y (target) variable in regression setting

You are right, it's not common at all, but what you do makes sense though it may be sensitive to your binning strategy. So, it's also a good idea to plot your binned target variable. This ...

### Why are my elastic net and lasso r-squared measures negative?

Dave has addressed your specific question and I suggest you accept that answer. The point about "unexpected performance" in machine learning is quite on target. What follows is more to get ...
Accepted

### Why are my elastic net and lasso r-squared measures negative?

PRESS is an out-of-sample measure. Even when you have an intercept, out-of-sample $R^2<0$ is possible. It means that you would have been better off predicting the mean of $y$ every time, regardless ...
Accepted

### How to interpret OOB Error in a Random Forest model

Yes, out-of-bag error is an estimate of the error rate (which is 1 - accuracy) that this training approach has for new data from the same distribution. This estimate is based on the predictions that ...

### What is the role of 'shuffle' in train_test_split()?

With time-series data, where you can expect auto-correlation in the data you should not split the data randomly to train and test set, but you should rather split it on time so you train on past ...
Accepted

### What model should I use to predict a time series like this?

I will be answering this as someone who has spent a LOT of time in the last few years working on financial time series prediction. However, I am not a professional (I do it on my own time etc.) ...

### Stratification of the continuous y (target) variable in regression setting

Update: First consider whether splitting the data into training and validation subsets makes the best use of your data for building a predictive model. Split-Sample Model Validation Bootstrap ...

### Shrink decision tree by shuffling order of attributes

As with any data problem, you should make sure your variables are correctly coded, e.g. as an unordered or ordered factor. Any good decision tree algorithm then will treat the gill-color variable as ...

### When does MAPE (Mean Absolute Percentage Error) fail?

MAPE can be problematic (see this thread for MAPE's shortcomings). In scikit-learn, if one of the values of ...

### What is the difference between pairwise kernels and pairwise distances?

I think to start; we should understand concerning Kernel(-functions): How to intuitively explain what a kernel is? Is a kernel function just a mapping? What is a kernel function? Pairwise distance ...
Accepted

### Prediction of Adaboost Classifier between 0 and 1

In sklearn you have predic_proba(X) function, which should give you the probability i.e output between 0-1 In ada_boost, this will be the weighted average of the probabilities of the classifiers in ...
Accepted

### Why do the roc_auc scores for train_test_split and for cross-validation differ so much?

Ok. I got it. It's my roc_auc_score. The correct code should use pipe.predict_proba(X_test).

CCA has inconsistent nomenclature; there are a few things that I saw being called loadings: variable weights or parameters that multiply your data and create the canonical variates. Analogous to beta^...

### Silhouette was not returning a valid number on scikit-learn on iris data. Is this wrong?

There is nothing "wrong" with it. First of all, R's and Python's implementations of the algorithm may differ, hence they may give different results. Second, $k$-means is a randomized ...
Accepted

### What is the interpretation of sklearn's linear perceptron coefficients?

The problem here lies in the hyperparameters. perc.n_iter_ is 6 after running your code; according to the defaults in the api, the default value for ...
Accepted

### Why do gradient boosting algorithms mostly use trees?

Mostly because they are very good base learner. In few words, I woud say because it is easy to boost trees, and the performance (in terms of predictive power) is very good. Usually, data mining ...

### How to get RandomForest model accuracy per value in Python, with categorical y-values?

No you can't calculate the metrics per each value. Say that $y_{56}$ is $0$ and the model predicted $1$, you cannot calculate the metrics based on single value. Technically you could say that the ...
Accepted

### Logistic regression- is it okay to build a model that maximizes recall and use the coefficients for inference

Maximum likelihood estimation exists for a reason. The gold standard objective function--the one that results in the best coefficients--is the log likelihood plus the penalty function. The penalty ...

### In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

It's correct that when the sample size is fixed, there is not a difference between the two statements of the optimization problem. Your demonstration in the revised question makes it clear that the ...

### XGBRegressor score (R2) vs. eval_metric (RMSE)

This is a bit strange, but $R^2$ and $RMSE$ are just functions of each other, so they are, in some sense, conveying the same information.  R^2=1-\dfrac{ n\big(RMSE\big)^2 }{ \sum_{i=1}^n\big( y_i-\...

### sklearn's permutation_importance returns surprising result

Permutation importance as implemented by Scikit for a linear model is based on the variance explained $R^2$ which is affected by both the coefficients and the variance of the variable underlying them. ...
Accepted

### Why are sklearn's cross_val_score values not increasing with the size of the training set?

I don't think this result is too surprising. Each of the points in your plot has an associated error measurement associated with it. The overall number of holes only varies in a small range, so the ...

### Scikit-learn QuantileRegressor memory allocation error. No issue with statsmodel QuantReg with the same data

The sklearn QuantileRegressor class uses linear programming to solve the quantile regression problem which is much more computationally expensive than iterative reweighted least squares as used by ...
1 vote

### Why does classifier (XGBoost) "after PCA" runtime increase compared to "before PCA"

If your original data was relatively discrete (had many "ties" for some or all the features), then this is a likely outcome, even after reducing the number of features. In a tree model like ...
1 vote
Accepted

### How to compute a 'pair confusion matrix'?

You are probably studying approaches and measures to compare partitions. In particularly, clustering partitions. One of the approaches and a class of measures is based on the so called comembership (...
1 vote

### Random Forest Parameter Settings for Big Data

I see you have any questions. Q&A sites like this are usually not best suited for such cases, I'd encourage you to ask one question at a thread next time. Also given the number of questions, it'd ...

Only top scored, non community-wiki answers of a minimum length are eligible