Questions tagged [data-leakage]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
9
votes
4answers
217 views

Combining PCA, feature scaling, and cross-validation without training-test data leakage

The sci-kit learn documentation for cross-validation says the following about using feature-scaling and cross-validation: Just as it is important to test a predictor on data held-out from training,...
1
vote
1answer
30 views

Data leakage if I add prediction as feature?

I have a training set and a test set. Let's assume the following: I train random forest on the training set I make prediction on training set and test set Then I add those prediction as features back ...
0
votes
1answer
17 views

Normalization of training and test set with data leakage

I have a time series data set for actual number of airport passengers. Within 15 years (2004 ~ 2019), just like having a trend, number of the passengers is increasing over time as the country is ...
0
votes
1answer
11 views

Standardize data before plotting learning curve

I have implemented cross validation function with hyper parameter tuning. Basically, doing the following: Split the data into 80% training, 20% testing apply cross validation with hyper parameter ...
0
votes
1answer
15 views

Is using the target value in sample weights target leakage?

If I'm training a regression model, and I want to weight the importance of each sample, is using (a function of) the target as the weight considered target leakage? Does this depend on the particular ...
0
votes
0answers
7 views

Data Leakage Question - Deriving Aggregates from Data Leakage Values

I am tasked with building a predictive model to predict flight delays with one year's worth of data from here: http://stat-computing.org/dataexpo/2009/the-data.html In order to avoid data leakage I ...
1
vote
1answer
51 views

Preventing information leakage when scaling a time series?

I have a time series $S_i$ that I want to train a regressor on to predict the next point in the time series. I want to split the data into training and validation sets, and also scale the data in the ...
0
votes
0answers
16 views

Will historical data lead to target leakage?

I'm bulding a employee churn model. I've employee data from 2016 to 2019 (of people who stayed/left the company), my goal is to train using data from 2016 to 2018 and predict on 2019. Since there's ...
0
votes
0answers
14 views

Data Leak during data acquisition for credit scoring

I have a few questions about data leaks. Particularly, I'm interested in a credit scoring data can have leakages. I'm at the stage of data acquisition and I suppose I have target leak but not sure. ...
1
vote
1answer
138 views

One hot encoding vs apply the average of the label to each category

I have a fairly reasonably sized dataset (row>50k). And I'm looking for the best way to utilize some of the categorical columns. For purpose of this question, let's say that one of the categorical ...
4
votes
1answer
567 views

Using Random Forest variable importance for feature selection

I'm currently trying to convince my colleague that his method of doing feature selection is causing data leakage and I need help doing so. The method they are using is as follows: They first run a ...
0
votes
0answers
22 views

Data leakage in multilevel validation

I participate in competition that have historical data. I break it down according to this scheme. ...
0
votes
1answer
107 views

Avoiding information leakage in CV folds with scaling

Chapter 6 (Algorithm Chains and Pipelines) in the book Introduction to ML with Python made me aware of a common mistake when scaling data for cross validation: leaking information into the test set by ...
3
votes
0answers
32 views

Is it okay to include the dependent variable as an input variable to the higher-level regression model, in a hierarchical / multi-level setup

Let's say I have a hierarchical dataset with student scores (for each student) nested within schools. While modelling for a varying intercept, would it be okay to include the average of student scores ...
1
vote
1answer
246 views

One-Hot Encoding and Feature Engineering While Avoiding Data Leakage

I have a Pandas dataframe for which I've performed some actions over categorical features: Feature Engineering One-Hot Encoding Let's say that in my dataset I have the features "person_income" and "...
0
votes
1answer
443 views

Avoiding Data Leakage using Cross Validation during aggregation

This question is very conceptual in nature so if it is too general please close it I have a fictitious dataset. The data set consists of 10k students across 5 schools It consists of an id, attrib_1, ...
1
vote
2answers
70 views

What is the correct approach in this case for modelling data (logistic regression)

I'm making a logistic regression model but am unsure about whether it is right or not to do the following: I'm trying to predict if a person will buy a high cost hotel, given by ...
3
votes
1answer
1k views

When imputing missing values in a test set, should the new values come from the training set or be recalculated from the test set?

Both answers to this question on imputing missing values note that, when imputing missing values in a test set for model evaluation, the replacement values should be the ones calculated and used in ...
2
votes
0answers
81 views

Data leakage concern in a binary classification problem

I have a binary classification problem (where 1 = broken and 0 = not broken) for machine engines under study. There are 25 continuous features over which I use to make predictions of 1 or 0 using ...
4
votes
0answers
43 views

Can SVM leak training data?

Is it possible to have access to trained model, e.g. through some API, and reverse engineer the model by asking for predictions for some arbitrary data, therefore recover the support vectors of the ...