# Questions tagged [scikit-learn]

A machine-learning library for Python. Use this tag for any on-topic question that (a) involves scikit-learn either as a critical part of the question or expected answer, & (b) is not just about how to use scikit-learn.

1,682 questions
Filter by
Sorted by
Tagged with
1 vote
13 views

### How to find the optimal coefficients of the two predict_proba output matrices of two different classifiers using regression and maximizing accuracy?

I am performing classification, where there are six labels and two predict_proba (predicted probabilities) matrices as outputs. These two predict_proba matrices correspond to the outputs of two ...
• 11
1 vote
27 views

### Lasso regression prediction on test set is predicting towards the mean of the train set?

I am using lasso regression to predict age (continuous data) from a set having 2112 numeric features (indepedent variable). The training dataset contains around 2773 participants. The mean of that ...
• 111
82 views

### In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
• 31
1 vote
10 views

### Scikit-learn's Gaussian Processes Classifier: Specific kernels for specific features [closed]

Using GaussianProcessClassifier from sklearn, is it possible to specify different kernels for different features? For example, $X$ is an n X 2 matrix and I would to use the RBF for the first column ...
• 6,360
60 views

### Number of samples in scikit-Learn cost function for Ridge/Lasso regression

I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this: , whereas the cost function for e.g. Ridge Regression is ...
12 views

### Difference K-fold versus Blocked Cross-Validation?

In the paper "Evaluating time series forecasting models: an empirical study on performance estimation methods" by Cerqueira et al (2020), they mention k-fold cross-validation. Which they ...
• 141
228 views

### Stratification of the continuous y (target) variable in regression setting

Is it wise to stratify the continuous y (target) variable when you split your training and testing data from the total sample in regression setting? Here is the approach in python to do implement ...
• 131
1 vote
10 views

### is nrmse scale-dependent?

Im trying to evaluate my regression models using a normalised version of the RMSE, nrmse = rmse(y, y_pred)/rmse(y, y_mean) where y_mean is the array of the same len as y filled with the mean value of ...
• 11
1 vote
30 views

### LinearRegression in Pytorch and sklearn, what is the differnece?

I am currently implementing Linear Regression in Pytorch and sklearn and I get two different Mean squared error (MSE) values for both. MSE is lower for Pytorch Linear Regression. Wanted to ask what ...
• 21
1 vote
17 views

### Avoiding data leakage in preprocessing and handling unseen values in test data

I've been reading up on avoiding data leakage in the preprocessing step of a machine-learning/data-science pipeline, specifically that it is wrong to apply preprocessing to both training and test data ...
• 111
21 views

### Using target values in both (X and y) arguments of fit(X, y)

This question is based on SQL Server Machine Learning Services Ski Rental tutorial. We have a dataframe df: ...
• 101
1 vote
47 views

### Linear Regression with Lasso Regularization by using scikitlearn and scipy.optimize

i am trying to apply lasso linear regression with both scikitlearn and scipy.optimize min method. However, i cannot reach same result. Code that i created with scipy.optimize can't shrink redundant ...
• 13
23 views

### evaluating scoring metrics during hyperparameter tuning

I'm struggling with a couple of concepts related to hyperparameter tuning. I'm developing a model (gradient boosted tree) in python using sklearn. Currently, I'm in the process of using sklearn's ...
42 views

### Python KernelPCA inverse_transform does not remove noise as it does with regular PCA function [closed]

Context: I am attempting noise reduction using PCA before entering features into a model. To do this, I am transforming the features into their principal components, removing the principal components ...
• 1
15 views

### what results are still usable from non-nested cross validation when tuning hyperparameters (and reconciling that with Optuna)

I'm trying to wrap my head around nested cross validation for the purposes of hyperparameter tuning. Part i) If I was to run hyperparameter tuning with cross validation (without nesting), say with ...
• 101
34 views

### class_weight='balanced' vs high f_beta score for imbalanced logistic regression in sklearn. Please help explain the difference

I have an imbalanced binary classification problem I am trying to solve with the LogisticRegression algorithm in sklearn. As the data is highly imbalanced I am looking at ways to treat the imbalance ...
24 views

### Derivatives of output w.r.t input on a neural network trained with standardized data

I'm using a neural network to model an unknown function for which I would also like to know the derivatives. The nn has four inputs and four outputs, and the training data is preprocessed using scikit-...
• 25
20 views

### Sequential feature selection stopping condition

When using sequential feature selection approach, say forward feature selection, I want to stop adding new features when the improvement in model scores is smaller than a certain level. Is there a ...
1 vote
43 views

### How is Cholesky decomposition used in ridge regression?

As far as I learnt, Cholesky decomposition can be used only for symmetrical positive definite matrices, but I can see it is used as solver in Sklearn-Ridge package, can somebody explain how it is used ...
• 11
11 views

### Minimize risk and add rejection to model

I want to minimize the risk of a Gaussian model with a cost for false negatives and false positives. The model uses Naive Bayes algorithm and solves a binary classification problem: P(x_i \mid y) = \...
14 views

### How do I evaluate if my data represent the target variable before training a machine learning algorithm?

I have a dataset of points cloud where each point in the point cloud has a variable. I am trying to relate the local geometry features to that point variable by using FPFH, This means I am generating ...
23 views

### Finding line equation that fits decision boundary

Given: A train and testing dataset with two features $x_1$ and $x_2$ Two binary classes for classification ($0$ or $1$) Training and prediction with the Naive Bayes algorithm for Gaussian ...
1 vote
14 views

### Train a Final Machine Learning Model with Tensorflow

Based on a previous question and on this article, it is suggested that you split the data between train and test (or train/validate/test). But once you have control of your model, you should retrain ...
• 127
14 views

### Difference between balanced_accuracy score and macro_averaged recall

I understand balanced_accuracy_score metrics metric is recommended as against accuracy_score in imbalanced learning. But one thing I find strange is this measure is always equal to the macro-averaged <...
• 231
23 views

### Inverse scaling of coefficient using SkLearn

I had constructed a simple Multiple linear regression model, where I have 2 independent variables and a target (dependent variable). Now, I transformed my independent variable using ...
28 views

### How is the train_score from sklearn.model_selection.cross_validate calculated?

I split the data 80/20 as follows: from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) ...
• 101
26 views

### How to implement logistic regression deviance from scratch

As a learning exercise, I'm trying to implement the deviance for logistic regression from scratch. I understand the deviance to be: $\mathcal{L}_S - \mathcal{L}_M$, where $\mathcal{L}_s$ is equal to ...
30 views

### How to forecast actual future values using XGBoost?

So I have a solar Irradiation dataset having around 61000+ rows & 2 columns. I have made the model using XGBoost to predict future values. I have split the data in 2 parts train and test and ...
1 vote
37 views

### Logistic regression- is it okay to build a model that maximizes recall and use the coefficients for inference

I'm a novice in the field of ML and stats. So I have a dataset where the target feature (dependent variable) is binary (True, False), I'm trying to make some inferences and find features in the ...
• 13
19 views

### My random forrest regressor was overfitting so I tried to use randomsearchcv but I still got a worse result, what should I change? [duplicate]

I tried to fix my overfitting with randomized search cross-validation. These are my params: I set 100 estimators but that is irrelevant for the overfitting. I read log2 was best for regressors ...
41 views

### Feature selection with RandomForest and then retrain RandomForest using the selected features

I am trying to classify patients into 2 different groups using a random forest. The features correspond to the gene expression of individual patients. This means, that I have around 20.000 features (...
• 101
1 vote
36 views

### sklearn vs RtSNE : Number of PCA components to retain in tSNE [closed]

In the R implementation you can pass the number of components to keep in PCA step. I cannot figure out if this is possible in sklearn implementation. Is it possible or I am missing it? Thank you! R: ...
• 13
1 vote
26 views

### I have set of features to relate to two different values. When I made a regressor for only one it worked well but if i use two it does not?

I have a set of 33x1 features (x) and they can be related to different two values in (y) and I have 1203985 observations. Using np.shape() you can see the dimensions of x and y. x= (1203985, 33) y=(...
5 views

### Why does sklearn list weighted precision and micro precision separately if they are the same thing?

This post explains that micro precision is the same as weighted precision. (And the logic applies to recall and f-score as well.) So why does sklearn.metrics list micro and weighted as separate ...
• 55
1 vote
24 views

### Timeseries forecassting (Load forecasting) - Apparent shift in actual vs predicted values when applying regression model

Tools/languages/techniques I am using python scikit-learn different regression models (only linear regression is shown here for simplicity) I am working on a regression problem. The data I have is ...
58 views

### How to handle missing values NaiveBayes Scikit Learn

I am working with a dataset which has 34 features (numerical, nominal) and the target class. Several of the columns have missing values, especially one column has approximately 50% missing values. I ...
15 views

### Optimum bandwidth for mean shift clustering

How do we decide what is the optimum bandwidth for mean shift algorithmn given our data? Take this for example - A demo of the mean-shift clustering algorithm via https://scikit-learn.org/stable/...
• 23
38 views

### DBSCAN clustering - epsilon

How do we decide on the optimum epsilon and min_samples to be specified given our data? Take this for example - Demo of DBSCAN clustering algorithm via https://scikit-learn.org/stable/auto_examples/...
• 23
34 views

### Number of coefficients and intercepts in sklearn logistic regression

I noticed that the matrix of coefficients learned by a logistic regression model (which can be retrieved with the .coef_ attribute) is $(c, n)$ where $n$ is the ...
• 55
31 views

### Strange cluster assignment at edge of cluster

I'm experimenting with HDBSCAN and am encountering some results that I don't fully understand. Mainly about a datapoint that gets assigned to another cluster than I would expect it to be. Hopefully ...
36 views

### Is it logical to combine cross-validation estimator like RidgeCV with cross_val_score in sklearn?

I was going through solutions for a regression problem competition on Kaggle here. Many solutions for the problem are combining cross-validation estimators like RidgeCV, LassoCV with cross_val_score ...
19 views

### When we use StandardScaler() in Pipeline with GridSearchCV, does it automatically take care of data leakage in k-fold CV?

A lot of tutorials use pipeline with GridSearchCV. Example here. ...
40 views

### Help Understanding Polynomial/Least Squares Regression

I have a dataset of 2 variables (called x with shape n x 2 values of x1 and x2) and 1 output (called y). I am having trouble understanding how to calculate predicted output values from the polynomial ...
13 views

### How to assess the significance of a single data point in Multidimensional scaling?

I'm looking for a way to determine a Stress-like value associated with the single data points of a Multidimensional scaling plot. The source of my data is a dissimilarity matrix from which I computed ...
57 views

### Using k-fold cross-validation of random forest: how many samples are used to create a tree?

I'm trying to tune the hyperparameters of my RandomForestRegressor created in python with sklearn with bootstrap = True using <...
• 21
24 views

### How do interpret this result for t-SNE plot results?

I am trying to interpret this plot I created for a malware dataset. The dataset contains Benign and Malware data. I am having a hard time understanding what it means. Any advice or help would be ...
17 views

### Looking for a scientific paper that explains the OneVsRestClassifier from scikit-learn

In my Logistic Regression Model I am using (among others) the OneVsRestClassifier, as the simplest tool to get probabilities for a multiclass problem. I understand how it works and I understand that ...
34 views

### Why are my random forest regression predicts valid probability distributions?

I have tabular input data where the labels correspond a probability distribution on five actions, E.g. a row might look like: x_0, x_1, ...., x_n, .1, .1, .3, .0, .5 I am using sklearn's Random forest ...