Skip to main content

Questions tagged [concept-drift]

In the context of data streams concept drift describes the phenomenon of the underlying distribution of a variable changing over time, often negatively affecting the performance of statistical models trained on perviously observed data.

Filter by
Sorted by
Tagged with
2 votes
1 answer
49 views

What if instead re-training our classification model, we only adjust the probability threshold?

As far as i know theoretically our model tend to be drifting/shifting as time goes on and need to be retrained. i wonder if its acceptable that instead of retraining the classification model, we keep ...
raffo's user avatar
  • 23
1 vote
1 answer
93 views

What's the best clustering algorithm for Fraud Data?

Background I'm working on a Fraud dataset with 500,000 samples, and 130 features. There are: 98 numerical features, 32 categorical features, There are missing values in: 7 numerical features, 12 ...
Connor's user avatar
  • 655
2 votes
0 answers
54 views

what's the model/data drift metric for time series forecasting?

Suppose that I have time series forecasting model, e.g. forecasting point of sales revenue in various economic scenarios represented by indicators such as inflation or interest rates. I build a model $...
Aksakal's user avatar
  • 61.9k
5 votes
2 answers
3k views

How do you find the KL Divergence between two multi-variable datasets?

Background I'm working on a tabular data model that performs a binary classification. The model has recently started underperforming and I'd like to know if that's due to a drift in the feature ...
Connor's user avatar
  • 655
0 votes
0 answers
121 views

How to distinguish between data drift and concept drift in data?

Our team is working on a regression-like model to predict a continuous value across time. We noticed that the model performance was quite good for part of the period, while deteriorate quite a lot at ...
Student's user avatar
  • 375
1 vote
0 answers
19 views

Determine drift

I would like to determine potential drift in the data/device. Is there a general acknowledged procedure and/or method to do so? I know how to visually "detect" a drift but I would like to ...
Ben's user avatar
  • 3,533
1 vote
1 answer
119 views

Why can't I detect concept drift with linear regression?

When I was first trying to detect concept drift, it seemed naively to me to be a problem of detecting whether noisy data was veering off horizontal trajectory (non-trivial slope above some arbitrary ...
Sanger Steel's user avatar
1 vote
0 answers
18 views

What is the term for the same ROCAUC performance between groups but different sensitivity at the same operating threshold?

I have a model that predicts disease in Group A with a ROCAUC of 0.90. That same model also predicts disease in Group B with a ROCAUC of 0.90. However, at an operating threshold of 0.5, the ...
superkayrad's user avatar
2 votes
0 answers
52 views

Identifying concept drift in the presence of data drift

Usually concept drift is defined as a change in the conditional distribution of our target variable with our covariates, $P_{0}(Y|X) \neq P_{t}(Y|X)$. It is also known as real concept drift or ...
Kozolovska's user avatar
  • 1,355
3 votes
0 answers
102 views

How can you determine whether there is concept drift or whether a model is affecting the distribution of the target class?

Assume that I am building a churn prediction model, and I collect observational data of customers who registered in the last 12-18 months. Assume that 50% of customers churned. Customers who are ...
Jay Ekosanmi's user avatar
10 votes
1 answer
4k views

Difference between distribution shift and data shift, concept drift and model drift

Lately, I am seeing both terms used interchangeably in several scenarios. Joaquin Quiñonero in MIT press (NIPS), Dataset Shift in ML NeurIPS 2021 workshop in DistShift Model drift: Towards Data ...
Carlos Mougan's user avatar
0 votes
0 answers
25 views

problems while production the clustering results

I searched for clustering in production but do not find related practical answers. Is it possible to make the clustering code in production? Suppose I have a data set for 1M users with around 100 ...
newleaf's user avatar
  • 101
2 votes
0 answers
34 views

Active learning to counter concept drift

I'll be doing my thesis soon on model drift detection and possible remedies in a production environment. I'll probably be making an intuitive (hopefully!) theoretical framework with various types of ...
Zestar75's user avatar
2 votes
0 answers
80 views

Find a representative samples from an estimated distribution by KDE

I served a Neural Network model trained on a huge (timeseries) dataset. In production, I would like to monitor the newly received data and check if there is a drift in the features using K-S testing. ...
Coderji's user avatar
  • 71
0 votes
1 answer
216 views

Find optimal training dataset after concept drift

There are many strategies how to detect a concept drift or model drift, like when there was a major change in the underlying process so that the model becomes invalid. It can be an abrupt change or it ...
HansHupe's user avatar
  • 153
1 vote
3 answers
2k views

Explain "concept drift" and how we can detect it in text data

Can we detect concept drift in text data?. I am dealing with text classification problem. If we can, how can we detect concept drift in text data?
Anees's user avatar
  • 13
1 vote
0 answers
55 views

Data ageing and concept drift

I have a simple question. In the literature I am reading "data ageing" and "concept drift" are mentioned, and I just want to make sure that I understand the difference between the two terms. The ...
Michael's user avatar
  • 56
0 votes
1 answer
156 views

Zero Denominator in Yule's Q-statistics?

In Concept Drift Adaptation by Exploiting Historical Knowledge they use Yule's Q-statistics to compute diversity between a collection of predictions. I think the context is not really important. I ...
gmp's user avatar
  • 1
7 votes
1 answer
6k views

How does LightGBM deals with incremental learning (and concept drift)?

With some research I found that it updates the leaves (does not create new or remove old ones) is it right? How this happens? Another question is when the incremental learning is done in concept ...
Jader Martins's user avatar
2 votes
0 answers
372 views

Explanation(s) for unimodal distribution of prediction probability computed by Random Forest

I have a typical binary classification problem with a sample of ~700 instances where I fitted multiple classification models including logistic regression, SVM and Random Forest. The instances are ...
Nmws's user avatar
  • 21
1 vote
0 answers
487 views

How to update a keras LSTM weights to avoid Concept Drift

I´m trying to update a Keras LSTM to avoid the concept of drift. For that I´m following the approach proposed in this paper [1] on which they compute an anomaly score and they apply it to update the ...
kevin's user avatar
  • 111
2 votes
0 answers
203 views

Disadvantages of "moving window ensemble" approach?

Assuming online/incremental training is not available for a particular algorithm, and assuming that you have a stream of training data that may or may not change over time (eg log data), what are the ...
dvas0004's user avatar
1 vote
1 answer
68 views

Concept drift in in user interaction data

concept drift usually refers to the change in the relationship between input and output data over time. I do have dataset of users' activity in an e-commerce website. Let's say we have a sequence of ...
miro's user avatar
  • 131
0 votes
1 answer
878 views

What is the corresponding model when I include drift in an SARIMA model with seasonal differences?

I fitted the following model, but I am not sure if the drift term is an intercept term or the mean of $y_t-y_{t-4}$ (as it seems in a blog post by Rob Hyndman). ...
Lane's user avatar
  • 1
0 votes
0 answers
230 views

Is this an example of overfitting?

I am trying to predict some future values using either KNN or regression model. I have about 9 independent variables that do not seem to have strong correlation to each other (Not completely sure ...
jwsong21's user avatar
3 votes
1 answer
73 views

Are ensemble learning methods for data streams restricted to online or batch learning?

Recently I'm working on some online learning algorithm (using RBF neural network ) for classification. As I read papers in this area I found there is an issue in online-learning called concept drift ...
mkafiyan's user avatar
  • 267
2 votes
1 answer
513 views

Unsupervised concept drift detection in clustering

I am implementing a clustering-based algorithm for non-stationary data stream. Most concept drift techniques are based on change in classification output (or on its accuracy). Is their a way for ...
Michael's user avatar
  • 31
2 votes
1 answer
101 views

Optimization with concept drift, changing location of the optimum over time

Is there any research/literature on the topic of optimization where the location of the optimum changes over time? Nonetheless I am interested in an optimal solution at any given time. It's hard for ...
jakob-r's user avatar
  • 220
4 votes
2 answers
468 views

lifetime of fraud detection models

Suppose we are building/testing a fraud detection model for a specific credit card/ or a quick cash loan business. We have a lot of data to play with (say past 5years), and after careful preprocessing,...
user6396's user avatar
  • 463
4 votes
1 answer
126 views

Is there a way to adapt machine learning models knowing ex ante that distributions will shift?

I am currently working on a topic where I know that the distributions of the output and of the covariates will shift. I know for example that some covariates will at least follow the inflation rate. ...
LouisBBBB's user avatar
  • 303
4 votes
1 answer
203 views

Machine learning or statistical models that account for time evolution and underlying system changes

I wonder if there are some algorithms that can account for underlying system dynamics over time. One possible situation can be the following: in a ticket reporting data, a data point arrives when a ...
hurrikale's user avatar
  • 893
3 votes
1 answer
506 views

How do the terms non-stationarity, concept-drift and evolving data relate to each other?

I often see the terms non-stationarity, concept-drift and evolving data in the same context, as if they were interchangeable. Are they? Or is there some subtle nuance that I am missing?
au.re's user avatar
  • 131
1 vote
1 answer
494 views

Synthetic datasets for concept drifting data

Is there any synthetic / artificial datasets for concept drifting data? I want to visualize the performance of some clustering algorithm when data experiences concept drift and changes over time.
eLearner's user avatar
  • 115
2 votes
1 answer
828 views

concept drift detection

I'm working on a project that involves concept drift detection for a time series. Are there any well-known techniques/methods/algorithms that are known to be effective for this sort of problem? ...
Kashif's user avatar
  • 517
1 vote
1 answer
392 views

Statistics tests for monitor changes in data distribution (concept drift) without knowing real output

I'm currently struggling with concept drift problem in on-line learing. I read some papers "Ikonomovska Gama - Regression Trees from Data Streams with Drift Detection " and check their implementation ...
Mr Jedi's user avatar
  • 383
1 vote
4 answers
2k views

Benchmark Data-sets for Concept Drift where important predictors (independent variables) change with time or stream of observations [closed]

I'm currently searching the web and literature for streaming classification datasets with concept drift. I've found a number of synthetic datasets where over time the important predictors either ...
Andrew Cassidy's user avatar