Questions tagged [scikit-learn]

A machine-learning library for Python. Use this tag for any on-topic question that (a) involves scikit-learn either as a critical part of the question or expected answer, & (b) is not just about how to use scikit-learn.

Filter by
Sorted by
Tagged with
9
votes
2answers
15k views

Preprocess categorical variables with many values [duplicate]

I have a dataset that consists of only categorical variables and a target variable. I want to predict the (binary) target variable with the categorical variables. I am trying to do this in Python and ...
52
votes
5answers
93k views

How does one interpret SVM feature weights?

I am trying to interpret the variable weights given by fitting a linear SVM. (I'm using scikit-learn): ...
15
votes
2answers
7k views

Use of nested cross-validation

Scikit Learn's page on Model Selection mentions the use of nested cross-validation: ...
18
votes
3answers
7k views

How to get hyper parameters in nested cross validation?

I have read the following posts for nested cross validation and still am not 100% sure what I am to do with model selection with nested cross validation: Nested cross validation for model selection ...
4
votes
2answers
4k views

Why is my high degree polynomial regression model suddenly unfit for the data?

I'm building a ridge regression model in scikit-learn and trying to find the optimal degree polynomial to use. The data I'm working with is a fairly predictable time series of hourly traffic volumes, ...
73
votes
1answer
69k views

How to split the dataset for cross validation, learning curve, and final evaluation?

What is an appropriate strategy for splitting the dataset? I ask for feedback on the following approach (not on the individual parameters like test_size or ...
20
votes
2answers
41k views

Random forest is overfitting?

I'm experimenting with random forests with scikit-learn and I'm getting great results of my training set, but relatively poor results on my test set... Here is the problem (inspired from poker) which ...
11
votes
1answer
9k views

Why does statsmodels.api.OLS over-report the r-squared value?

I am using statsmodels.api.OLS to fit a linear regression model with 4 input-features. The shape of the data is: ...
6
votes
1answer
1k views

How is a ROCAUC=1.0 possible with imperfect accuracy? [duplicate]

I used sklearn to compute roc_auc_score for a dataset of 72 instances. The accuracy was at 97% (2 misclassifications), but the ROC AUC score was 1.0. How is this ...
62
votes
2answers
52k views

One-hot vs dummy encoding in Scikit-learn

There are two different ways to encoding categorical variables. Say, one categorical variable has n values. One-hot encoding converts it into n variables, while dummy encoding converts it into n-1 ...
11
votes
1answer
10k views

Scikit Binomial Deviance Loss Function

This is scikit GradientBoosting's binomial deviance loss function, ...
9
votes
6answers
7k views

Making a single decision tree from a random forest

I am using scikit learn to build a Random Forest classifier. I have heard that it might be possible to build a single decision tree from a Random Forest. The suggestion is that although the ...
9
votes
1answer
4k views

How to obtain optimal hyperparameters after nested cross validation?

In general, if we have a large dataset, we can split it into (1) training, (2) validation, and (3) test. We use validation to identify the best hyperparameters in cross validation (e.g., C in SVM) and ...
18
votes
3answers
6k views

Why not use the “normal equations” to find simple least squares coefficients?

I saw this list here and couldn't believe there were so many ways to solve least squares. The "normal equations" on Wikipedia seemed to be a fairly straight forward way: $$ {\displaystyle {\begin{...
10
votes
2answers
10k views

Implementation of nested cross-validation

I'm trying to figure out if my understanding of nested cross-validation is correct, therefore I wrote this toy example to see if I'm right: ...
15
votes
2answers
26k views

Using BIC to estimate the number of k in KMEANS

I am currently trying to compute the BIC for my toy data set (ofc iris (: ). I want to reproduce the results as shown here (Fig. 5). That paper is also my source for the BIC formulas. I have 2 ...
7
votes
3answers
17k views

How to deal with a skewed class in binary classification having many features?

I am doing data analysis in the mobile ad targeting domain. I have around 18 features and for a combination of these features, the result is either True or False (1/0) depending on whether the ...
15
votes
3answers
3k views

Methods to work around the problem of missing data in machine learning

Virtually any database we want to make predictions using machine learning algorithms will find missing values ​​for some of the characteristics. There are several approaches to address this problem, ...
11
votes
2answers
5k views

What are the differences between Ridge regression using R's glmnet and Python's scikit-learn?

I am going through the LAB section §6.6 on Ridge Regression/Lasso in the book 'An Introduction to Statistical Learning with Applications in R' by James, Witten, Hastie, Tibshirani (2013). More ...
6
votes
3answers
3k views

scikit-learn score metric on the coefficient of determination $R^2$

I am using scikit-learn in Python and they define a quantity called score. It's defined in the middle of the documentation page. Reproduced here: Returns the ...
6
votes
3answers
2k views

Raw data outperforms Z-score transformed data in SVM classification

I've been trying to perform a binary classification using an SVM classifier (scikit-learn's SVC with RBF kernel). I have a sample size of about 100, with about 70 features each. The features are of ...
43
votes
3answers
83k views

Logistic Regression: Scikit Learn vs Statsmodels

I am trying to understand why the output from logistic regression of these two libraries gives different results. I am using the dataset from UCLA idre tutorial, predicting ...
4
votes
2answers
12k views

Lasso cross validation

I want to perform cross validation to find the regularization parameter for Lasso. I am using scikit-learn library in python. I first generate the dataset and then perform k-fold cross-validation. ...
26
votes
2answers
11k views

Why is Python's scikit-learn LDA not working correctly and how does it compute LDA via SVD?

I was using the Linear Discriminant Analysis (LDA) from the scikit-learn machine learning library (Python) for dimensionality reduction and was a little bit curious ...
5
votes
1answer
22k views

Logistic regression and scaling of features

I was under the belief that scaling of features should not affect the result of logistic regression. However, in the example below, when I scale the second feature by uncommenting the commented line, ...
16
votes
2answers
4k views

Is decision threshold a hyperparameter in logistic regression?

Predicted classes from (binary) logistic regression are determined by using a threshold on the class membership probabilities generated by the model. As I understand it, typically 0.5 is used by ...
8
votes
4answers
1k views

One hot encoding of a binary feature when using XGBoost

I already asked this question is SO; however, I realized that this may be a better place for this type of question. I am well aware that when using categorical features with tree based models such as ...
8
votes
1answer
8k views

Where can I read about gamma coefficient in SVM in scikit-learn?

Scikit learn support vector machine algorithm have a couple of coefficients which meaning I can not understand. gamma : float, optional (default=0.0) Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’...
7
votes
2answers
978 views

Lasso penalty only applied to subset of regressors

This question has been asked before but there were no responses, so I thought I might ask again. I'm interested in applying a Lasso penalty to some subset of the regressors, i.e. with objective ...
7
votes
1answer
700 views

What makes a Random Forest random besides bootstrapping and random sampling of features?

After reading about random forests in the original paper and in textbooks I was under the impression that what makes the model random is bootstrapping - training each tree on a different random subset ...
5
votes
1answer
3k views

Why is ridge regression giving different results in Matlab and Python?

Why is the output from Matlab and Python vary for ridge regression? I use the ridge command in Matlab and scikit-learn in Python ...
10
votes
1answer
2k views

How to Compute the Brier Score for more than Two Classes

tl;dr How do I correctly compute the Brier score for more than two classes? I got confusing results with different approaches. Details below. As suggested to me in a comment to this question, I ...
10
votes
4answers
805 views

Combining PCA, feature scaling, and cross-validation without training-test data leakage

The sci-kit learn documentation for cross-validation says the following about using feature-scaling and cross-validation: Just as it is important to test a predictor on data held-out from training,...
8
votes
1answer
10k views

Why are all Lasso coefficients in model 0.0?

I'm using from sklearn.linear_model import Lasso in Python 2.7.6 I wrote a script that I've used for doing a Lasso regression for my Features (X) and my Targets (y)...
3
votes
2answers
5k views

Getting distance of points from decision boundary with linear SVM?

I posted this originally in Stack Overflow but realize it might be more of a statistics question. I am using SKLearn to run SVC on my data. ...
1
vote
2answers
2k views

Better accuracy with validation set than test set

I trained a model with some algorithms like random forest, logistic regression and so on. My dataset was split into 80% CV train data (so actually 60% of data to train the model and 20 % for testing ...
6
votes
2answers
4k views

Difference between Random forest vs Bagging in sklearn

I know that Random forest uses Bagging, but how does ensemble give a single prediction in sklearn Bagging vs Random forest? Also, I do not understand why in Bagging ...
5
votes
1answer
9k views

Log Loss function in scikit-learn returns different values

I have been trying to wrap my head around the log loss function for model evaluation. I understand how the value is calculated after doing the math by hand. In the python module ...
5
votes
1answer
4k views

Regression when the dependent variable is between 0 and 1

I am using the scikit-learn library to perform regression. However in my case I need the dependent variable to be constrained in the range 0 to 1. The dependent variable represents count proportions ...
2
votes
3answers
880 views

Sklearn - Choosing the right model for supervised learning/classification task

I am beginning to learn how to use scikit-learn and I have a hard time choosing the right model. Here is my dataset: I have 100 persons. Each person was measured ...
0
votes
1answer
46 views

In sklearn, it seems that `dot(x, x)` corresponds to `np.sum(X*X,axis=1)[:, np.newaxis]`, why is that? [closed]

In a 2d Euclidean space, Let point $a=(a_x, a_y)=(1,1)$; Let point $b=(b_x, b_y)=(5,1)$; Let point $c=(c_x, c_y)=(4,4)$; the squared Euclidean distance between point a and point b is equal to $(...
19
votes
2answers
22k views

Difference between selecting features based on “F regression” and based on $R^2$ values?

Is comparing features using F-regression the same as correlating features with the label individually and observing the $R^2$ value? I have often seen my ...
23
votes
3answers
54k views

How to systematically remove collinear variables (pandas columns) in Python? [closed]

Thus far, I have removed collinear variables as part of the data preparation process by looking at correlation tables and eliminating variables that are above a certain threshold. Is there a more ...
34
votes
3answers
91k views

Polynomial regression using scikit-learn

I am trying to use scikit-learn for polynomial regression. From what I read polynomial regression is a special case of linear regression. I was hopping that maybe one of scikit's generalized linear ...
15
votes
1answer
18k views

What is the difference between decision_function, predict_proba, and predict function for logistic regression problem?

I have been going through the sklearn documentation but I am not able to understand the purpose of these functions in the context of logistic regression. For ...
37
votes
1answer
67k views

what does the numbers in the classification report of sklearn mean?

I have below an example I pulled from sklearn 's sklearn.metrics.classification_report documentation. What I don't understand is why there are f1-score, precision and recall values for each class ...
15
votes
7answers
9k views

Random forest is overfitting

I am trying to use Random Forest Regression in scikits-learn. The problem is I am getting a really high test error: train MSE, 4.64, test MSE: 252.25. This is how ...
20
votes
4answers
28k views

Difference between statsmodel OLS and scikit linear regression

I have a question about two different methods from different libraries which seems doing same job. I am trying to make linear regression model. Here is the code which I using statsmodel library with ...
12
votes
4answers
41k views

Principal Component Analysis and Regression in Python

I'm trying to figure out how to reproduce in Python some work that I've done in SAS. Using this dataset, where multicollinearity is a problem, I would like to perform principal component analysis in ...
14
votes
1answer
4k views

Nystroem Method for Kernel Approximation

I have been reading about the Nyström method for low-rank kernel aproximation. This method is implemented in scikit-learn [1] as a method to project data samples to a low-rank approximation of the ...