Questions tagged [scikit-learn]
A machine-learning library for Python. Use this tag for any on-topic question that (a) involves scikit-learn either as a critical part of the question or expected answer, & (b) is not just about how to use scikit-learn.
1,760
questions
0
votes
0
answers
193
views
class_weight='balanced' vs high f_beta score for imbalanced logistic regression in sklearn. Please help explain the difference
I have an imbalanced binary classification problem I am trying to solve with the LogisticRegression algorithm in sklearn.
As the data is highly imbalanced I am looking at ways to treat the imbalance ...
2
votes
1
answer
224
views
Derivatives of output w.r.t input on a neural network trained with standardized data
I'm using a neural network to model an unknown function for which I would also like to know the derivatives. The nn has four inputs and four outputs, and the training data is preprocessed using scikit-...
- 25
1
vote
0
answers
142
views
Sequential feature selection stopping condition
When using sequential feature selection approach, say forward feature selection, I want to stop adding new features when the improvement in model scores is smaller than a certain level. Is there a ...
- 11
1
vote
0
answers
150
views
How is Cholesky decomposition used in ridge regression?
As far as I learnt, Cholesky decomposition can be used only for symmetrical positive definite matrices, but I can see it is used as solver in Sklearn-Ridge package, can somebody explain how it is used ...
- 11
1
vote
1
answer
223
views
Train a Final Machine Learning Model with Tensorflow
Based on a previous question and on this article, it is suggested that you split the data between train and test (or train/validate/test). But once you have control of your model, you should retrain ...
- 157
4
votes
1
answer
1k
views
Inverse scaling of coefficient using SkLearn
I had constructed a simple Multiple linear regression model, where I have 2 independent variables and a target (dependent variable).
Now, I transformed my independent variable using ...
0
votes
1
answer
1k
views
How is the train_score from sklearn.model_selection.cross_validate calculated?
I split the data 80/20 as follows:
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
...
- 101
0
votes
0
answers
395
views
How to implement logistic regression deviance from scratch
As a learning exercise, I'm trying to implement the deviance for logistic regression from scratch.
I understand the deviance to be:
$\mathcal{L}_S - \mathcal{L}_M$, where $\mathcal{L}_s$ is equal to ...
0
votes
0
answers
1k
views
How to forecast actual future values using XGBoost?
So I have a solar Irradiation dataset having around 61000+ rows & 2 columns. I have made the model using XGBoost to predict future values.
I have split the data in 2 parts train and test and ...
- 1
1
vote
1
answer
61
views
Logistic regression- is it okay to build a model that maximizes recall and use the coefficients for inference
I'm a novice in the field of ML and stats.
So I have a dataset where the target feature (dependent variable) is binary (True, False), I'm trying to make some inferences and find features in the ...
- 13
0
votes
0
answers
22
views
My random forrest regressor was overfitting so I tried to use randomsearchcv but I still got a worse result, what should I change? [duplicate]
I tried to fix my overfitting with randomized search cross-validation. These are my params:
I set 100 estimators but that is irrelevant for the overfitting. I read log2 was best for regressors ...
- 15
0
votes
1
answer
124
views
Feature selection with RandomForest and then retrain RandomForest using the selected features
I am trying to classify patients into 2 different groups using a random forest. The features correspond to the gene expression of individual patients. This means, that I have around 20.000 features (...
- 101
1
vote
1
answer
187
views
sklearn vs RtSNE : Number of PCA components to retain in tSNE [closed]
In the R implementation you can pass the number of components to keep in PCA step.
I cannot figure out if this is possible in sklearn implementation.
Is it possible or I am missing it?
Thank you!
R: ...
- 13
1
vote
1
answer
29
views
I have set of features to relate to two different values. When I made a regressor for only one it worked well but if i use two it does not?
I have a set of 33x1 features (x) and they can be related to different two values in (y) and I have 1203985 observations. Using np.shape() you can see the dimensions of x and y.
x= (1203985, 33) y=(...
- 15
0
votes
1
answer
155
views
Why does sklearn list weighted precision and micro precision separately if they are the same thing?
This post explains that micro precision is the same as weighted precision.
(And the logic applies to recall and f-score as well.)
So why does sklearn.metrics list micro and weighted as separate ...
- 137
2
votes
1
answer
289
views
Timeseries forecassting (Load forecasting) - Apparent shift in actual vs predicted values when applying regression model
Tools/languages/techniques I am using
python
scikit-learn
different regression models (only linear regression is shown here for simplicity)
I am working on a regression problem. The data I have is ...
0
votes
1
answer
440
views
How to handle missing values NaiveBayes Scikit Learn
I am working with a dataset which has 34 features (numerical, nominal) and the target class. Several of the columns have missing values, especially one column has approximately 50% missing values.
I ...
- 1
1
vote
0
answers
546
views
DBSCAN clustering - epsilon
How do we decide on the optimum epsilon and min_samples to be specified given our data? Take this for example - Demo of DBSCAN clustering algorithm via https://scikit-learn.org/stable/auto_examples/...
- 33
2
votes
1
answer
768
views
Number of coefficients and intercepts in sklearn logistic regression
I noticed that the matrix of coefficients learned by a logistic regression
model (which can be retrieved with the .coef_ attribute) is $(c, n)$
where $n$ is the ...
- 137
2
votes
1
answer
223
views
Is it logical to combine cross-validation estimator like RidgeCV with cross_val_score in sklearn?
I was going through solutions for a regression problem competition on Kaggle here. Many solutions for the problem are combining cross-validation estimators like RidgeCV, LassoCV with cross_val_score ...
- 31
0
votes
1
answer
561
views
When we use StandardScaler() in Pipeline with GridSearchCV, does it automatically take care of data leakage in k-fold CV?
A lot of tutorials use pipeline with GridSearchCV. Example here.
...
- 105
0
votes
1
answer
127
views
Help Understanding Polynomial/Least Squares Regression
I have a dataset of 2 variables (called x with shape n x 2 values of x1 and x2) and 1 output (called y). I am having trouble understanding how to calculate predicted output values from the polynomial ...
- 11
2
votes
2
answers
3k
views
Using k-fold cross-validation of random forest: how many samples are used to create a tree?
I'm trying to tune the hyperparameters of my RandomForestRegressor created in python with sklearn with bootstrap = True using <...
- 21
0
votes
0
answers
144
views
How do interpret this result for t-SNE plot results?
I am trying to interpret this plot I created for a malware dataset. The dataset contains Benign and Malware data. I am having a hard time understanding what it means. Any advice or help would be ...
0
votes
1
answer
236
views
Why are my random forest regression predicts valid probability distributions?
I have tabular input data where the labels correspond a probability distribution on five actions, E.g. a row might look like:
x_0, x_1, ...., x_n, .1, .1, .3, .0, .5
I am using sklearn's Random forest ...
1
vote
0
answers
279
views
How can I make sense of alpha values obtained from scikit-learn OC-SVM?
I am building a ML model that uses the OC-SVM for anomaly detection. For our cost function we require the alphas obtained from the OC-SVM. We use the OC-SVM of scikit-learn, which I assume is based on ...
- 11
1
vote
0
answers
41
views
Working with sparse data in numpy and sklearn [closed]
I have a time series dataset geenrated from some electrophysiological data. I have a frequncy dataset and the matrix is quite sparse but huge, like it contains 0.005 s time bins for 2000+ neurons ...
- 368
2
votes
0
answers
233
views
Log transformed response variable and overfitting issues in random forest regression
I am trying to fit random forest regression for continuous skewed response variable with so many zeros. Here is the plot of the response variable:
In An Introduction
to Statistical
Learning, authors ...
- 307
0
votes
0
answers
214
views
Breaking Tie with Ward's Method Hierarchical Clustering
Viewing the set of single-feature observations below, I think its obvious that the appropriate Euclidean distance-based (e.g., Ward's method) number of clusters is 3.
...
- 227
1
vote
1
answer
36
views
Learning-Agnostic Evaluation of SVM Models
I am at a point where I want to evaluate existing SVM models.
For this task I assume I have:
SVM model (to make it easier let's say it's a scikit-learn one)
Training Dataset that was used to learn (1)...
- 11
1
vote
0
answers
313
views
Why the grid scores from RFECV using ROC AUC is different from ROC AUC obtained by the same selected features?
I'm using RFECV with the scoring ROC AUC for feature selection and the model selected 3 features. However, when use these 3 features with the same estimator and same cross validation when I run the ...
- 11
1
vote
0
answers
53
views
Predicting negative class rather than target in SVM [closed]
I am trying to classify a target group from controls using a SVM. I am predicting probabilities, and noticed that when predicting the target class, the SVM performance was horrible (AUC ~0.2). This ...
- 11
1
vote
0
answers
241
views
Should I have more trees than dimensions for the Isolation Forest?
I have a dataset which has 200 dimensions after pre-processing. I read multiple times that 100 is the recommended number of trees for the Isolation Forest. Since each tree chooses one feature randomly,...
- 31
1
vote
1
answer
163
views
Is there an incremental dimensionality reduction algorithm that can handle batch size less than number of components to be reduced?
I have a large dataset of patient data by hour. For example, given the shape as (hours, features), patient 1 data shape could be ...
- 123
1
vote
0
answers
170
views
what is the RFE feature ranking based method of sci-kit learn?
I have used sci-kit learn RFE and RFECV to select the best features. https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.RFE.html, It gives the ranking of the features. can ...
- 11
1
vote
0
answers
40
views
Is using SVC for detection problems is the right choice?
I have an NLP problem(fake news detection) and I used the code below, can I use svc for classification? and is my cross-validation accurate?
...
- 11
0
votes
1
answer
249
views
How am I misunderstanding sklearn.metrics.roc_auc_score? [closed]
The documentation for scikit-learn sklearn.metrics.roc_auc_score() contains two statements regarding the 'average' parameter that, together, are confusing me:
Note: multiclass ROC AUC currently only ...
- 145
1
vote
2
answers
214
views
Why is Scikit's Support Vector Classifier returning support vectors with decision scores outside [-1,1]? Is this a mistake?
I'm currently playing around with support vector machines in Scikit Learn and I've come across some unusual behaviour. For a basic simulated dataset, I've trained an SVC estimator (with linear kernel),...
- 11
0
votes
1
answer
92
views
How to get RandomForest model accuracy per value in Python, with categorical y-values?
I'm using a Random Forest model in Python (sklearn) to predict categorical y-values using a X,y dataset that is split in training and a testing dataset. The model accuracy is calculated using the ...
- 109
1
vote
2
answers
338
views
When is it okay to accept overfit model for production?
I am working on a binary classification problem using random forests (75:25 class proportion).m label 0 is minority class. I am following the below approach
a) execute RF with default ...
- 2,522
0
votes
0
answers
62
views
Sklearn PCA projection differences while reproducing projection sample by sample
I've been stuck on this issue due to inability to reproduce the performance of PCA-classification pipeline on training data, but in case the pipeline receives one sample at a time. This seems to be ...
1
vote
0
answers
17
views
Linear Regression vs Keras [duplicate]
I created a dummy dataset and compared the performance of SKLearn LinearRegression and Keras.
Why is Keras producing horrible results compared to Linear Regression?
Code:
...
- 11
1
vote
0
answers
187
views
manually calculate classification report [closed]
I have a confusion matrix like as shown below
pred
0 1
0 17 38
act
1 16 174
I am trying to find the measures of ...
- 2,522
1
vote
1
answer
183
views
Why do gradient boosting algorithms mostly use trees?
Why do gradient boosting algorithms mostly use trees? Is there any logic in this? (in XGBoost and in boosting which in sklearn library uses trees, not other algorithms).
- 23
0
votes
0
answers
359
views
Normalization and RidgeCV in Sklearn Pipeline - possible data leakage?
To avoid data leakage between the train and test set, I'm using sklearn's Pipeline as follows:
...
- 1
1
vote
2
answers
114
views
Decision tree classification on Mushroom data
I have been going through notebooks available online where classification of mushrooms are done to poisonous(p) or edible(e) ...
- 161
1
vote
1
answer
370
views
What is the interpretation of sklearn's linear perceptron coefficients?
I'm stumped as to why this example doesn't do a better job fitting the data, I suspect it has to do with my interpretation of the perceptron object's coefficients. Note that I'm interested in the <...
- 191
1
vote
4
answers
2k
views
Making ensemble of K models trained during K-fold cross-validation
Let's suppose we are doing K-fold cross-valiation to estimate the performance of a model with a given set of hyperparameters.
...
- 145
0
votes
1
answer
221
views
Confusion while validating dataset using cross-validation in scikit learn
My confusion stems from this image
found on the Scikit-learn docs here. As far as I understand from this picture is that the entire dataset is split into two train and test sets and cross-validation ...
0
votes
0
answers
41
views
Mix of numerical and categorical features - to scale or not to scale?
I have the means of my features like this of an employee dataset:
...
- 111