Questions tagged [concept-drift]
In the context of data streams concept drift describes the phenomenon of the underlying distribution of a variable changing over time, often negatively affecting the performance of statistical models trained on perviously observed data.
30
questions
1
vote
0
answers
15
views
What is the term for the same ROCAUC performance between groups but different sensitivity at the same operating threshold?
I have a model that predicts disease in Group A with a ROCAUC of 0.90.
That same model also predicts disease in Group B with a ROCAUC of 0.90.
However, at an operating threshold of 0.5, the ...
- 31
1
vote
0
answers
15
views
Identifying concept drift in the presence of data drift
Usually concept drift is defined as a change in the conditional distribution of our target variable with our covariates,
$P_{0}(Y|X) \neq P_{t}(Y|X)$. It is also known as real concept drift or ...
- 1,035
3
votes
0
answers
57
views
How can you determine whether there is concept drift or whether a model is affecting the distribution of the target class?
Assume that I am building a churn prediction model, and I collect observational data of customers who registered in the last 12-18 months. Assume that 50% of customers churned. Customers who are ...
- 581
0
votes
0
answers
40
views
Threshold of KL divergence for detecting data drift
While I've come across posts introducing KL divergence as a mean for data drift detections. However, I fail to observe any on of them suggesting beyond what threshold value of KL divergence should we ...
- 135
8
votes
1
answer
1k
views
Difference between distribution shift and data shift, concept drift and model drift
Lately, I am seeing both terms used interchangeably in several scenarios.
Joaquin Quiñonero in MIT press (NIPS), Dataset Shift in ML
NeurIPS 2021 workshop in DistShift
Model drift: Towards Data ...
- 238
0
votes
0
answers
24
views
problems while production the clustering results
I searched for clustering in production but do not find related practical answers.
Is it possible to make the clustering code in production?
Suppose I have a data set for 1M users with around 100 ...
- 101
1
vote
0
answers
25
views
Active learning to counter concept drift
I'll be doing my thesis soon on model drift detection and possible remedies in a production environment. I'll probably be making an intuitive (hopefully!) theoretical framework with various types of ...
- 11
2
votes
0
answers
70
views
Find a representative samples from an estimated distribution by KDE
I served a Neural Network model trained on a huge (timeseries) dataset. In production, I would like to monitor the newly received data and check if there is a drift in the features using K-S testing. ...
- 71
0
votes
1
answer
166
views
Find optimal training dataset after concept drift
There are many strategies how to detect a concept drift or model drift, like when there was a major change in the underlying process so that the model becomes invalid. It can be an abrupt change or it ...
- 143
0
votes
3
answers
1k
views
Concept drift in text data
Can we detect concept drift in text data?. I am dealing with text classification problem. If we can,
how can we detect concept drift in text data?
- 3
0
votes
0
answers
41
views
Data ageing and concept drift
I have a simple question.
In the literature I am reading "data ageing" and "concept drift" are mentioned, and I just want to make sure that I understand the difference between the two terms. The ...
- 46
0
votes
1
answer
110
views
Zero Denominator in Yule's Q-statistics?
In Concept Drift Adaptation by Exploiting Historical Knowledge
they use Yule's Q-statistics to compute diversity between a collection of predictions.
I think the context is not really important.
I ...
- 1
5
votes
1
answer
2k
views
How does LightGBM deals with incremental learning (and concept drift)?
With some research I found that it updates the leaves (does not create new or remove old ones) is it right? How this happens?
Another question is when the incremental learning is done in concept ...
- 185
2
votes
0
answers
246
views
Explanation(s) for unimodal distribution of prediction probability computed by Random Forest
I have a typical binary classification problem with a sample of ~700 instances where I fitted multiple classification models including logistic regression, SVM and Random Forest.
The instances are ...
- 21
1
vote
0
answers
414
views
How to update a keras LSTM weights to avoid Concept Drift
I´m trying to update a Keras LSTM to avoid the concept of drift. For that I´m following the approach proposed in this paper [1] on which they compute an anomaly score and they apply it to update the ...
- 111
2
votes
0
answers
109
views
Disadvantages of "moving window ensemble" approach?
Assuming online/incremental training is not available for a particular algorithm, and assuming that you have a stream of training data that may or may not change over time (eg log data), what are the ...
- 21
1
vote
1
answer
38
views
Concept drift in in user interaction data
concept drift usually refers to the change in the relationship between input and output data over time.
I do have dataset of users' activity in an e-commerce website. Let's say we have a sequence of ...
- 131
0
votes
1
answer
342
views
What is the corresponding model when I include drift in an SARIMA model with seasonal differences?
I fitted the following model, but I am not sure if the drift term is an intercept term or the mean of $y_t-y_{t-4}$ (as it seems in a blog post by Rob Hyndman).
...
- 1
0
votes
0
answers
192
views
Is this an example of overfitting?
I am trying to predict some future values using either KNN or regression model.
I have about 9 independent variables that do not seem to have strong correlation to each other (Not completely sure ...
3
votes
1
answer
58
views
Are ensemble learning methods for data streams restricted to online or batch learning?
Recently I'm working on some online learning algorithm (using RBF neural network ) for classification. As I read papers in this area I found there is an issue in online-learning called concept drift ...
- 247
1
vote
1
answer
357
views
Unsupervised concept drift detection in clustering
I am implementing a clustering-based algorithm for non-stationary data stream.
Most concept drift techniques are based on change in classification output (or on its accuracy).
Is their a way for ...
- 21
1
vote
1
answer
78
views
Optimization with concept drift, changing location of the optimum over time
Is there any research/literature on the topic of optimization where the location of the optimum changes over time? Nonetheless I am interested in an optimal solution at any given time.
It's hard for ...
- 200
4
votes
2
answers
427
views
lifetime of fraud detection models
Suppose we are building/testing a fraud detection model for a specific credit card/ or a quick cash loan business. We have a lot of data to play with (say past 5years), and after careful preprocessing,...
- 423
4
votes
1
answer
114
views
Is there a way to adapt machine learning models knowing ex ante that distributions will shift?
I am currently working on a topic where I know that the distributions of the output and of the covariates will shift. I know for example that some covariates will at least follow the inflation rate. ...
- 193
4
votes
1
answer
178
views
Machine learning or statistical models that account for time evolution and underlying system changes
I wonder if there are some algorithms that can account for underlying system dynamics over time.
One possible situation can be the following: in a ticket reporting data, a data point arrives when a ...
- 873
2
votes
1
answer
287
views
How do the terms non-stationarity, concept-drift and evolving data relate to each other?
I often see the terms non-stationarity, concept-drift and evolving data in the same context, as if they were interchangeable. Are they? Or is there some subtle nuance that I am missing?
- 121
1
vote
1
answer
389
views
Synthetic datasets for concept drifting data
Is there any synthetic / artificial datasets for concept drifting data? I want to visualize the performance of some clustering algorithm when data experiences concept drift and changes over time.
- 105
2
votes
1
answer
790
views
concept drift detection
I'm working on a project that involves concept drift detection for a time series. Are there any well-known techniques/methods/algorithms that are known to be effective for this sort of problem?
...
- 457
1
vote
1
answer
368
views
Statistics tests for monitor changes in data distribution (concept drift) without knowing real output
I'm currently struggling with concept drift problem in on-line learing. I read some papers "Ikonomovska Gama - Regression Trees from Data Streams with Drift Detection " and check their implementation ...
- 373
1
vote
4
answers
2k
views
Benchmark Data-sets for Concept Drift where important predictors (independent variables) change with time or stream of observations [closed]
I'm currently searching the web and literature for streaming classification datasets with concept drift. I've found a number of synthetic datasets where over time the important predictors either ...
- 476