Questions tagged [concept-drift]

In the context of data streams concept drift describes the phenomenon of the underlying distribution of a variable changing over time, often negatively affecting the performance of statistical models trained on perviously observed data.

Filter by
Sorted by
Tagged with
1 vote
0 answers
15 views

What is the term for the same ROCAUC performance between groups but different sensitivity at the same operating threshold?

I have a model that predicts disease in Group A with a ROCAUC of 0.90. That same model also predicts disease in Group B with a ROCAUC of 0.90. However, at an operating threshold of 0.5, the ...
user avatar
1 vote
0 answers
15 views

Identifying concept drift in the presence of data drift

Usually concept drift is defined as a change in the conditional distribution of our target variable with our covariates, $P_{0}(Y|X) \neq P_{t}(Y|X)$. It is also known as real concept drift or ...
user avatar
  • 1,035
3 votes
0 answers
57 views

How can you determine whether there is concept drift or whether a model is affecting the distribution of the target class?

Assume that I am building a churn prediction model, and I collect observational data of customers who registered in the last 12-18 months. Assume that 50% of customers churned. Customers who are ...
user avatar
0 votes
0 answers
40 views

Threshold of KL divergence for detecting data drift

While I've come across posts introducing KL divergence as a mean for data drift detections. However, I fail to observe any on of them suggesting beyond what threshold value of KL divergence should we ...
user avatar
  • 135
8 votes
1 answer
1k views

Difference between distribution shift and data shift, concept drift and model drift

Lately, I am seeing both terms used interchangeably in several scenarios. Joaquin Quiñonero in MIT press (NIPS), Dataset Shift in ML NeurIPS 2021 workshop in DistShift Model drift: Towards Data ...
user avatar
0 votes
0 answers
24 views

problems while production the clustering results

I searched for clustering in production but do not find related practical answers. Is it possible to make the clustering code in production? Suppose I have a data set for 1M users with around 100 ...
user avatar
  • 101
1 vote
0 answers
25 views

Active learning to counter concept drift

I'll be doing my thesis soon on model drift detection and possible remedies in a production environment. I'll probably be making an intuitive (hopefully!) theoretical framework with various types of ...
user avatar
2 votes
0 answers
70 views

Find a representative samples from an estimated distribution by KDE

I served a Neural Network model trained on a huge (timeseries) dataset. In production, I would like to monitor the newly received data and check if there is a drift in the features using K-S testing. ...
user avatar
  • 71
0 votes
1 answer
166 views

Find optimal training dataset after concept drift

There are many strategies how to detect a concept drift or model drift, like when there was a major change in the underlying process so that the model becomes invalid. It can be an abrupt change or it ...
user avatar
  • 143
0 votes
3 answers
1k views

Concept drift in text data

Can we detect concept drift in text data?. I am dealing with text classification problem. If we can, how can we detect concept drift in text data?
user avatar
  • 3
0 votes
0 answers
41 views

Data ageing and concept drift

I have a simple question. In the literature I am reading "data ageing" and "concept drift" are mentioned, and I just want to make sure that I understand the difference between the two terms. The ...
user avatar
  • 46
0 votes
1 answer
110 views

Zero Denominator in Yule's Q-statistics?

In Concept Drift Adaptation by Exploiting Historical Knowledge they use Yule's Q-statistics to compute diversity between a collection of predictions. I think the context is not really important. I ...
user avatar
  • 1
5 votes
1 answer
2k views

How does LightGBM deals with incremental learning (and concept drift)?

With some research I found that it updates the leaves (does not create new or remove old ones) is it right? How this happens? Another question is when the incremental learning is done in concept ...
user avatar
2 votes
0 answers
246 views

Explanation(s) for unimodal distribution of prediction probability computed by Random Forest

I have a typical binary classification problem with a sample of ~700 instances where I fitted multiple classification models including logistic regression, SVM and Random Forest. The instances are ...
user avatar
  • 21
1 vote
0 answers
414 views

How to update a keras LSTM weights to avoid Concept Drift

I´m trying to update a Keras LSTM to avoid the concept of drift. For that I´m following the approach proposed in this paper [1] on which they compute an anomaly score and they apply it to update the ...
user avatar
  • 111
2 votes
0 answers
109 views

Disadvantages of "moving window ensemble" approach?

Assuming online/incremental training is not available for a particular algorithm, and assuming that you have a stream of training data that may or may not change over time (eg log data), what are the ...
user avatar
1 vote
1 answer
38 views

Concept drift in in user interaction data

concept drift usually refers to the change in the relationship between input and output data over time. I do have dataset of users' activity in an e-commerce website. Let's say we have a sequence of ...
user avatar
  • 131
0 votes
1 answer
342 views

What is the corresponding model when I include drift in an SARIMA model with seasonal differences?

I fitted the following model, but I am not sure if the drift term is an intercept term or the mean of $y_t-y_{t-4}$ (as it seems in a blog post by Rob Hyndman). ...
user avatar
  • 1
0 votes
0 answers
192 views

Is this an example of overfitting?

I am trying to predict some future values using either KNN or regression model. I have about 9 independent variables that do not seem to have strong correlation to each other (Not completely sure ...
user avatar
3 votes
1 answer
58 views

Are ensemble learning methods for data streams restricted to online or batch learning?

Recently I'm working on some online learning algorithm (using RBF neural network ) for classification. As I read papers in this area I found there is an issue in online-learning called concept drift ...
user avatar
  • 247
1 vote
1 answer
357 views

Unsupervised concept drift detection in clustering

I am implementing a clustering-based algorithm for non-stationary data stream. Most concept drift techniques are based on change in classification output (or on its accuracy). Is their a way for ...
user avatar
  • 21
1 vote
1 answer
78 views

Optimization with concept drift, changing location of the optimum over time

Is there any research/literature on the topic of optimization where the location of the optimum changes over time? Nonetheless I am interested in an optimal solution at any given time. It's hard for ...
user avatar
  • 200
4 votes
2 answers
427 views

lifetime of fraud detection models

Suppose we are building/testing a fraud detection model for a specific credit card/ or a quick cash loan business. We have a lot of data to play with (say past 5years), and after careful preprocessing,...
user avatar
  • 423
4 votes
1 answer
114 views

Is there a way to adapt machine learning models knowing ex ante that distributions will shift?

I am currently working on a topic where I know that the distributions of the output and of the covariates will shift. I know for example that some covariates will at least follow the inflation rate. ...
user avatar
  • 193
4 votes
1 answer
178 views

Machine learning or statistical models that account for time evolution and underlying system changes

I wonder if there are some algorithms that can account for underlying system dynamics over time. One possible situation can be the following: in a ticket reporting data, a data point arrives when a ...
user avatar
  • 873
2 votes
1 answer
287 views

How do the terms non-stationarity, concept-drift and evolving data relate to each other?

I often see the terms non-stationarity, concept-drift and evolving data in the same context, as if they were interchangeable. Are they? Or is there some subtle nuance that I am missing?
user avatar
  • 121
1 vote
1 answer
389 views

Synthetic datasets for concept drifting data

Is there any synthetic / artificial datasets for concept drifting data? I want to visualize the performance of some clustering algorithm when data experiences concept drift and changes over time.
user avatar
  • 105
2 votes
1 answer
790 views

concept drift detection

I'm working on a project that involves concept drift detection for a time series. Are there any well-known techniques/methods/algorithms that are known to be effective for this sort of problem? ...
user avatar
1 vote
1 answer
368 views

Statistics tests for monitor changes in data distribution (concept drift) without knowing real output

I'm currently struggling with concept drift problem in on-line learing. I read some papers "Ikonomovska Gama - Regression Trees from Data Streams with Drift Detection " and check their implementation ...
user avatar
  • 373
1 vote
4 answers
2k views

Benchmark Data-sets for Concept Drift where important predictors (independent variables) change with time or stream of observations [closed]

I'm currently searching the web and literature for streaming classification datasets with concept drift. I've found a number of synthetic datasets where over time the important predictors either ...
user avatar