All Questions
Tagged with outliers time-series
155 questions
2
votes
3
answers
41
views
Testing forecasting accuracy - outliers [ with example]
I have a simple model that produces forecast values. The model works on hourly data. Now, I am only interested in observations with flags. I would like to identify where the forecasts are ...
- 79
1
vote
1
answer
85
views
Local Outlier Factor for time series
I hope this makes sense. I have discovered LOF and tried it in R. However, since I am dealing with time series, the neighbors cannot be "future" neighbors of the current observation(s). I am ...
- 307
1
vote
1
answer
272
views
to determine the appropriate threshold of the z-score for the non-normally distributed data
I am interested in CPI. And I need to identify outliers in the series. For that, my instructor mentioned about the number of standard deviations from the mean that a data point is. This is Z-score.
I ...
- 1,152
2
votes
2
answers
487
views
Methods for Detecting outliers in a time series
I have a question on detecting the outliers in a time series like PPI, CPI, inflation,...etc.)
Which method should I use? How can I precisely detect these outliers in a test or a method?
Please ...
1
vote
1
answer
41
views
Wrong time series detection
I have a problem and I need help.
I have a time series and I need to know if the data is correct. Let me explain with an example. Suppose I have data generated by an atmospheric pressure sensor. The ...
- 141
2
votes
1
answer
38
views
ARIMA - Identifying an outlier in residuals
I am trying to perform an ARIMA (SARIMAX in fact) and when looking at the residuals I see a large outlier. I am using python statsmodels.tsa.statespace.sarimax.
I ...
- 123
0
votes
0
answers
21
views
Statistical procedure to remove a time series which is an "outlier" in a set of replicated time series
This data are from measuring optical density in a bacterial growth experiment. These correspond to 4 time series which are biological replicates of exactly the same treatment (with the label 0_4)
The ...
0
votes
1
answer
102
views
Anomaly Detection in Multivariate and Univariate timeseries
I just started exploring Anomaly detection in timeseries for Univariate, Multivariate timeseries.
I read few articles about it, few research papers as well. But every article/research paper has ...
- 33
0
votes
0
answers
31
views
What is it called when an outlier falls out of a rolling window statistical calculation?
I have a time series $X_t \sim N(0, 1)$. There is a single outlier at index 347, at 8.5 standard deviations from the mean. If I now compute a rolling window standard deviation of $X_t$ with window ...
- 1,334
0
votes
1
answer
838
views
How to deal with Covid outlier in time series/machine learning forecasting?
Disclaimer: I checked some similar questions but I could not find anything in particular that would work for my case.
I am dealing with a time series going from 2015 to 2023. The data points are the ...
- 11
0
votes
1
answer
42
views
Evaluate outliers of strictly non-decreasing sequences
Say I have the following sequence:
Is there a way to get a probability for each point indicating whether it is an outlier or not of the underlining strictly non-decreasing sequence?
I suppose the ...
- 141
1
vote
0
answers
32
views
Which metric for neural network should I try for time series data with sudden peaks?
I am doing time series forecasting with neural network (feedforward for now, but I will test also RNNs) and my problem is that, even though the network learned general patterns, it doesn't forecast ...
- 115
0
votes
0
answers
22
views
How to impute additive outliers in time series data
I need to forecast daily electricity demand. It seems that the outliers in my dataset are additive as they are affected by an anomalous behavior and are not induced by a random process that also ...
- 291
0
votes
1
answer
142
views
What is a suitable technique for detecting anomalies in time series data?
I have a problem, where I try to identify if a machine performs an activity when it is not supposed to, or performs it an unusual number of times.
I am attempting to this using an anomaly detection ...
- 33
0
votes
0
answers
13
views
Identify outliers of river levels that change continuously over time
This is a time-dependent measure of the water level of a river measured by an instrument that measures the water level every five minutes. However, due to some interference and other factors, there ...
2
votes
0
answers
347
views
How to detect low and high flow outliers with seasonal time series data in R?
I have a dataset recording daily river flow from 1976 to 2017. I want to find out unusually high (potential flood) or low (potential drought) flow values from that datatset. What's the best way to ...
- 181
3
votes
1
answer
578
views
Why am I getting strange upper & lower limits on a gamma distribution?
I am working on a time series dataset. I understand it has a gamma distribution. I want to use a 99% probability threshold to establish upper & lower limits/cut-offs and find anomalies. However, I ...
- 31
1
vote
1
answer
478
views
Detecting outliers in a multiple time-series
I have some broker prices incoming in real-time for several products. Sometimes a broker makes a typo and sends a wrong price, or my parsing engine assigns the price to the wrong product - these are ...
- 369
1
vote
1
answer
204
views
Detecting Spikes in a 1-D discrete time series data with unknown underlying distribution
I have a discrete 1-D data set with a value range of 0-100. The underlying distribution is unknown --although we have enough data to fit a model-- to summarize it is a highly right-skewed data set, ...
- 111
2
votes
1
answer
339
views
Flagging bad time series behavior (Pattern Recognition and Outlier Detection)
I want to get some opinions on how to approach the following problem to do with detecting "unhealthy" behavior in time series data (either using a statistical/analytical model or ML/DL, I do ...
- 49
0
votes
1
answer
207
views
Heteroskedastic time series outlier analysis using machine learning
Is anyone aware of machine learning models that are able to deal with heteroskedasticity in time series, when trying to detect outliers? There are a lot of anomaly detection tools out there (like k-...
- 65
1
vote
1
answer
305
views
detecting outliers in weight measurement
I have weights data of users collected over a period of time. My goal is to find incorrect weight readings. The definition of incorrect readings is purely based on logical reasons (or in other words ...
- 121
1
vote
1
answer
297
views
Method for outlier detection in noisy seasonal time series data?
I have around 1000 times series of around 1000 samples, where each sample is 5 minutes a part.
An example of a time series after performing seasonal decomposition is
As we can see the data is very ...
- 171
0
votes
0
answers
60
views
Relative anomalies in multiple multivariate times series with different lengths?
I have a set of time series, highly correlated (similar peaks and trend). I'm going to find relative anomalies, e.g. say there are 20 times series. At a snapshot date, all values increase, but one ...
- 11
1
vote
1
answer
576
views
Scientific name of rolling median applied for times series outliers detection algorithm
I am a beginner in Machine Learning. I have a small python module related to times series outliers detection. I found in some posts and blogs an algorithm using thresholds calculated based on the ...
- 11
4
votes
1
answer
276
views
Matrix Profile and mean-shift detection
I'm currently working on anomaly detection on time series and one of the discords I'm trying to detect are 'mean-shifts,' i.e. the signal suddenly shifting by a certain value while retaining its ...
- 41
1
vote
1
answer
163
views
Statistical significance of outlier in time series
I have a time series for which I observe a new data point. I have reasons to suspect it to be an outlier (unsurprisingly, because of the COVID pandemic). Visually, this is confirmed. What I am ...
- 249
2
votes
2
answers
902
views
Optimal window size for contextual outlier detection
I am looking for methods to detect univariate contextual outliers in time series data. One example application is data from industrial plants in different (unknown) operation modes or slow trends or ...
- 153
0
votes
0
answers
19
views
Best way to compute statistics for normal upper and lower temperatures of a cooler
I have lots of temperature data and when plotted it shows events like door open for a while and the repetitive cycling of the compressor every few minutes. I would like to calculate Means for ...
- 101
2
votes
0
answers
55
views
Testing if a day of data is an outlier
I have a time series which exhibits a relatively similar behavior from one day to another. I want to know if a specific day of the time series is an outlier in the sense that it significantly differs ...
- 21
0
votes
1
answer
2k
views
Can I remove outliers from a residual plot? Or does this compromise the validity of my model?
I used the function auto.arima to predict sales for the next year. When using only 3 years of the dataset, my results were not good. When I go back 10 years, it improved.
However, in order for me to ...
- 103
1
vote
1
answer
2k
views
How does forecast::tsclean() detect outliers in R?
Does it use a particular z-score?
I know that it does apply STL.
My data is seasonal, and had quite a few outliers, so I am just wondering how exactly it determined whether a particular data point is ...
- 145
0
votes
0
answers
352
views
Smoothing train/test data
I am currently working on time series forecasting. I know that the first step is to divide the time series into train and test. Then I also understand that I have to normalize the test set using the ...
0
votes
0
answers
47
views
Anomaly Detection for Shifting Patterns
Suppose I build a model(e.g. Mahalanobis distance) to learn the normal pattern shown in green dots of the below plot. Then it is deployed into the production to detect anomalies. But as daily customer ...
- 1,027
1
vote
1
answer
92
views
What's the best way to find outliers in the time-series, encountering that it is a real-world mechanical process (process continuity)?
What's the best way to find outliers in the time-series, encountering continuity?
I attached two time-series that I'm interested to filter. One is less noisy, and one is a bit noisier. I'm mostly ...
- 11
1
vote
1
answer
3k
views
A Robust Technique to Detect Outliers from Time Series Data
I want to extract outliers from time series data which contains faulty values. I want to filter out outliers from stock prices. I have around 14,000 such timeseries and I can't analyse them manually. ...
- 147
1
vote
0
answers
35
views
tsoutliers functionality
the tso function of the tsoutliers package locate the outliers and and shows a potential outliers you might have but i noticed that from the plot that my data might have a lot more than that, is that ...
- 11
1
vote
0
answers
25
views
Timeseries analysis on current data
Please forgive the newbie nature of this question, but I'm really struggling with Mr Google.
The vast majority of resources I can find on time-series analysis seem to be concerned with the Holy Grail ...
- 171
2
votes
1
answer
887
views
Algorithms for Anomaly Detection of Event Sequence Data [Python/R]
I am building an anomaly detection system of event sequence data (transactions). For each timestep, a transaction can be in any of 76 different stages.
My dataset is therefore a 3D array of size(m,t,N)...
- 171
0
votes
2
answers
1k
views
Time Series Decomposition: Is it necessary (or wise) to remove outliers beforehand?
Do outliers change the outcome of time series decomposition?
As far as I understand it, outliers occur in the residual-component. In the residuals plot they can be visually identified as spikes.
...
- 341
1
vote
1
answer
611
views
How to handle outliers/spikes in time series and machine learning models when the number of observations is lower than expected?
I'm not that familiar with time series models, but I wanted some help to understand if there is a technique to handle outliers in periods where there are small number of observations.
For instance, if ...
- 111
1
vote
0
answers
132
views
Determine Significance of single observation compared to past average
Say I have count data per month. I want to compare a single month's count to the rest of the dataset and determine for a given month, is that month significantly different than the rest of the data?
...
- 111
0
votes
1
answer
159
views
Simulation study for anomaly detection methods in time series
Im trying to compare some anomaly detection methods for time series. Since there are not many time series with labeled outliers I decided to do a simulation study.
I'm doing the simulation in R using ...
2
votes
1
answer
91
views
Electrical Consumption Outlier Detection
Suppose you have several years of monthly consumption (kWh) data for 500,000 electrical meters and your job is to look for outlier behavior of various types. How would you approach modeling the meters ...
- 23
6
votes
2
answers
4k
views
Detect abrupt change in time series
I am trying to detect abrupt change (the "bump") in my data. My end goal is to fit a decline curve that describes the overall trend of a gas well's production rate over time. When fitting my curve, I ...
- 1,071
3
votes
0
answers
91
views
Resources for learning the time series stuff they don’t (or didn’t) teach you
I at one point, a long time ago, had two years of graduate econometrics focusing on time series, plus more on micro cross-section techniques. I haven’t made much use of the time-series stuff for a ...
2
votes
1
answer
228
views
Detecting outliers in time-series if I don't have a "normal" dataset [duplicate]
I have been trying to detect anomalies in my time-series dataset. What I am trying to accomplish is the following: I have a multivariate dataset, where two columns are of special interest. One tells ...
- 121
1
vote
1
answer
79
views
How to handle timeseries extremes (sigma > 20) in deep learning?
I'm using 16-channel, 400-Hz, standardized EEG data to train CNN-LSTM for seizure classification. The data contains $O(3)$ sigma > 20 points, rarely thousands in a ...
1
vote
1
answer
72
views
What are the anomalies/fault/outliers detection algorithms
I'm working on a weather application that uses data coming from multiple sensors in real time (the data is time series), i've made an anomalies detection model using One Class Support Vector Machines, ...
- 21
3
votes
2
answers
2k
views
How to identify outliers in a time series with correlated variables
I am working with time series data of sensor measurements. I have nine sensors that are in the same ballpark location recording the same data every 10 minutes. The sensors are setup such that the ...
- 133