7 votes

Why using StandardScaler from sklearn before LogisticRegression increase avg cross_val_score? Standarization should only help in faster convergece

By default, sklearn logistic regression is penalized. From the documentation: penalty{‘l1’, ‘l2’, ‘elasticnet’, ‘none’}, default=’l2’ Specify the norm of the penalty: ...
user avatar
  • 80.3k
5 votes
Accepted

How to find nearest neighbors using cosine similarity for all items from a large embeddings matrix?

Actually, we can use cosine similarity in knn via sklearn. The source code is here. This works for me: ...
user avatar
  • 66
5 votes
Accepted

Why does univariate Mahalanobis distance not match z-score?

Your intuition about the Mahalanobis distance is correct. However, the EllipticEnvelope algorithm computes robust estimates of the location and covariance matrix which don't match the raw estimates. (...
user avatar
  • 4,162
4 votes

Suitable approach to cluster histogram-like dataset using HDBSCAN implementation in python

Don't ask about the algorithm: focus on solving your problem. The peak-finding solution I posted at https://stats.stackexchange.com/a/428083/919 will help you analyze the situation and decide how many ...
user avatar
  • 294k
4 votes

Stratification of the continuous y (target) variable in regression setting

You are right, it's not common at all, but what you do makes sense though it may be sensitive to your binning strategy. So, it's also a good idea to plot your binned target variable. This ...
user avatar
  • 52.5k
3 votes

Why are my elastic net and lasso r-squared measures negative?

Dave has addressed your specific question and I suggest you accept that answer. The point about "unexpected performance" in machine learning is quite on target. What follows is more to get ...
user avatar
  • 66.4k
3 votes
Accepted

Why are my elastic net and lasso r-squared measures negative?

PRESS is an out-of-sample measure. Even when you have an intercept, out-of-sample $R^2<0$ is possible. It means that you would have been better off predicting the mean of $y$ every time, regardless ...
user avatar
  • 33.5k
3 votes
Accepted

How to interpret OOB Error in a Random Forest model

Yes, out-of-bag error is an estimate of the error rate (which is 1 - accuracy) that this training approach has for new data from the same distribution. This estimate is based on the predictions that ...
user avatar
  • 23.4k
3 votes

What is the role of 'shuffle' in train_test_split()?

With time-series data, where you can expect auto-correlation in the data you should not split the data randomly to train and test set, but you should rather split it on time so you train on past ...
user avatar
  • 117k
3 votes
Accepted

What model should I use to predict a time series like this?

I will be answering this as someone who has spent a LOT of time in the last few years working on financial time series prediction. However, I am not a professional (I do it on my own time etc.) ...
user avatar
3 votes

Stratification of the continuous y (target) variable in regression setting

Update: First consider whether splitting the data into training and validation subsets makes the best use of your data for building a predictive model. Split-Sample Model Validation Bootstrap ...
user avatar
  • 4,162
2 votes

Shrink decision tree by shuffling order of attributes

As with any data problem, you should make sure your variables are correctly coded, e.g. as an unordered or ordered factor. Any good decision tree algorithm then will treat the gill-color variable as ...
user avatar
2 votes

When does MAPE (Mean Absolute Percentage Error) fail?

MAPE can be problematic (see this thread for MAPE's shortcomings). In scikit-learn, if one of the values of ...
user avatar
2 votes

What is the difference between pairwise kernels and pairwise distances?

I think to start; we should understand concerning Kernel(-functions): How to intuitively explain what a kernel is? Is a kernel function just a mapping? What is a kernel function? Pairwise distance ...
user avatar
  • 385
2 votes
Accepted

Prediction of Adaboost Classifier between 0 and 1

In sklearn you have predic_proba(X) function, which should give you the probability i.e output between 0-1 In ada_boost, this will be the weighted average of the probabilities of the classifiers in ...
user avatar
  • 1,033
2 votes
Accepted

Why do the roc_auc scores for train_test_split and for cross-validation differ so much?

Ok. I got it. It's my roc_auc_score. The correct code should use pipe.predict_proba(X_test).
user avatar
2 votes

Cannot manually reproduce CCA loadings

CCA has inconsistent nomenclature; there are a few things that I saw being called loadings: variable weights or parameters that multiply your data and create the canonical variates. Analogous to beta^...
user avatar
  • 6,494
2 votes

Silhouette was not returning a valid number on scikit-learn on iris data. Is this wrong?

There is nothing "wrong" with it. First of all, R's and Python's implementations of the algorithm may differ, hence they may give different results. Second, $k$-means is a randomized ...
user avatar
  • 117k
2 votes
Accepted

What is the interpretation of sklearn's linear perceptron coefficients?

The problem here lies in the hyperparameters. perc.n_iter_ is 6 after running your code; according to the defaults in the api, the default value for ...
user avatar
  • 1,214
2 votes
Accepted

Why do gradient boosting algorithms mostly use trees?

Mostly because they are very good base learner. In few words, I woud say because it is easy to boost trees, and the performance (in terms of predictive power) is very good. Usually, data mining ...
user avatar
2 votes

How to get RandomForest model accuracy per value in Python, with categorical y-values?

No you can't calculate the metrics per each value. Say that $y_{56}$ is $0$ and the model predicted $1$, you cannot calculate the metrics based on single value. Technically you could say that the ...
user avatar
  • 117k
2 votes
Accepted

Logistic regression- is it okay to build a model that maximizes recall and use the coefficients for inference

Maximum likelihood estimation exists for a reason. The gold standard objective function--the one that results in the best coefficients--is the log likelihood plus the penalty function. The penalty ...
user avatar
2 votes

In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

It's correct that when the sample size is fixed, there is not a difference between the two statements of the optimization problem. Your demonstration in the revised question makes it clear that the ...
user avatar
  • 80.3k
2 votes

XGBRegressor score (R2) vs. eval_metric (RMSE)

This is a bit strange, but $R^2$ and $RMSE$ are just functions of each other, so they are, in some sense, conveying the same information. $$ R^2=1-\dfrac{ n\big(RMSE\big)^2 }{ \sum_{i=1}^n\big( y_i-\...
user avatar
  • 33.5k
2 votes

sklearn's permutation_importance returns surprising result

Permutation importance as implemented by Scikit for a linear model is based on the variance explained $R^2$ which is affected by both the coefficients and the variance of the variable underlying them. ...
user avatar
  • 689
2 votes
Accepted

Why are sklearn's cross_val_score values not increasing with the size of the training set?

I don't think this result is too surprising. Each of the points in your plot has an associated error measurement associated with it. The overall number of holes only varies in a small range, so the ...
user avatar
  • 80.3k
2 votes

Scikit-learn QuantileRegressor memory allocation error. No issue with statsmodel QuantReg with the same data

The sklearn QuantileRegressor class uses linear programming to solve the quantile regression problem which is much more computationally expensive than iterative reweighted least squares as used by ...
user avatar
  • 51
1 vote

Why does classifier (XGBoost) "after PCA" runtime increase compared to "before PCA"

If your original data was relatively discrete (had many "ties" for some or all the features), then this is a likely outcome, even after reducing the number of features. In a tree model like ...
user avatar
  • 3,046
1 vote
Accepted

How to compute a 'pair confusion matrix'?

You are probably studying approaches and measures to compare partitions. In particularly, clustering partitions. One of the approaches and a class of measures is based on the so called comembership (...
user avatar
  • 52.9k
1 vote

Random Forest Parameter Settings for Big Data

I see you have any questions. Q&A sites like this are usually not best suited for such cases, I'd encourage you to ask one question at a thread next time. Also given the number of questions, it'd ...
user avatar
  • 117k

Only top scored, non community-wiki answers of a minimum length are eligible