All Questions
Tagged with online-algorithms time-series
36 questions
1
vote
0
answers
71
views
Improve HMM state estimation in latest data
I have a time-series dataset that is poisson-distributed, where each day I get a new additional datapoint. If I input all the data into a HMM (I am using code I found from hmmlearn in python) it does ...
- 91
0
votes
1
answer
130
views
Time series model in production - Re-train on the fly as as batch process?
Let's say I've a time series of phone calls per day over the last three years. I could train a model using exponential smoothing (e.g. HoltWinters) for predicting the future amount of phone calls per ...
4
votes
1
answer
2k
views
Live peak / trough detection (data provided)
At the bottom of this question is the data of three time series in CSV-format. All are of same length and they all contain measurements of the same event "A". But each time series is using a ...
- 91
0
votes
1
answer
94
views
Classification of Imbalanced and Streaming Time Series Data
I have a question about classification of time series. Data has two features and I want to classify it into 5 classes. We have a stream of data and new data is generated every 5 seconds. Moreover in ...
- 1
1
vote
0
answers
44
views
Which metrics should be used in preprocessing in continual learning?
So my idea is to train an LSTM - autoencoder for anomaly detection by continual learning, i.e., I want to update the model after each 10 time steps. Firstly I will train it on source data, then re-...
- 753
0
votes
1
answer
106
views
Forecasting in time series (ARMA, GARCH etc.)
I've read here https://otexts.com/fpp2/arima-forecasting.html how we do forecasting in time series models like the ARMA model, but I'm wondering if we recalculate estimates of parameters of our model ...
- 97
5
votes
2
answers
254
views
What are good resources for online time series forecasting? [closed]
I have a project in which I'm given the state of the order book for a stock every 1ms, and I need to predict the return on the stock 2 minutes in the future using this information. I haven't been able ...
4
votes
0
answers
362
views
Continuous time series classification with lstm in Keras?
I have been researching time series classification with LSTM. I've seen examples where they provide continuous predictions, i.e. the prediction is updated at each time step. Is it possible to train a ...
- 1,239
3
votes
1
answer
980
views
Efficient online (rolling window) estimation of a GARCH model
I have a time series $x_t$ of length $n$. I would like to model it using rolling window approach with window length (width) $w$:
window $1$: $x_1,\dots,x_w$,
window $2$: $x_2,\dots,x_{w+1}$,
$\dots$,
...
- 69.5k
1
vote
0
answers
35
views
Online estimation of drifting discrete probability
I recently come across (in a practical setting) to the following problem. Suppose I receive items from a finite set ,one at a time . At each moment one item is drawn independently from an unknown ...
- 201
0
votes
1
answer
116
views
Real-Time signal processing with RandomForestClassifier in sklearn always predicts one class
I am trying to perform real-time decision making on data from a radar sensor and trying to detect occupancy. I generated data using the same sensor annotated it manually as vacant or occupied. I ...
- 103
1
vote
0
answers
236
views
Online deseasonalization of time series data
Are there any existing methodologies of deseasonalizing time series data online, in order to avoid lookahead bias? It seems that if you don't deaseasonalize time series data online, you would not be ...
- 159
3
votes
1
answer
566
views
Algorithms for real-time classification of segments in noisy time-series data
I’m trying to detect various features of a toy train track while driving on it:
The primary input is data from an optical sensor. The following image shows the recorded signal when driving over the ...
- 131
4
votes
1
answer
129
views
Latest development in online learning and causality inference
The context is this - I'm considering doing a part time PhD in statistical learning and today I've met up with a prospective supervisor who suggested that I think about causality in machine learning ...
- 257
1
vote
0
answers
47
views
Predicting user selections based on previous answers and datetime
I'm completely new to machine learning and wish to implement it into my app to help my users travel between places. Let's say I have data (constantly updating data) that looks like this:
...
- 111
2
votes
0
answers
313
views
Adjust decay rate dynamically
Say I have a stream of values $\langle s_1, s_2,\ldots\rangle$ coming in and a function
$$E_{s_1:s_n}(t) = E_{s_1:s_{n-1}}(t-1) + \alpha\cdot (s_t-E_{s_1:s_{n-1}}(t-1))$$
that compute their ...
- 121
4
votes
1
answer
55
views
What are some instructive examples of machine learning reproducing the behaviour of simulators?
I have a computer model which reproduce the behaviour of a physical system we want to control. The model includes a bit of fluid dynamics, heat exchange, pressure calculations etc. Inputs include both ...
1
vote
1
answer
56
views
A way to estimate if a random variable has shifted beyond a threshold
I have a random variable estimated over time by an online algorithm. I have the mean and variance of the random Gaussian variable at every step t. I expect the time series to have sudden shifts. What ...
- 11
6
votes
1
answer
5k
views
Online learning in LSTM
Recently, I have been working on RNNs (LSTM specifically) to do time series prediction and I have used different frameworks such as deeplearning4j and theano (keras). As you may know, one of the ...
- 366
2
votes
0
answers
319
views
Forecasting multivariate time series data stream
I have a multivariate time series data stream. I am looking for a method that can forecast the next value of one of the variables as the data comes in. (It would be a major advantage if there's an R ...
- 149
4
votes
0
answers
150
views
textbooks/literature/resources for online learning/ time series/ sequential analysis? [closed]
I have read (the contents page of) machine learning introductory textbooks such as Pattern Recognition and Machine Learning, Machine Learning A Probabilistic Perspective, and The Elements of ...
- 16.8k
0
votes
4
answers
4k
views
online detection of plateaus in time series
I need to detect plateaus in time series data online. The data I am working with represents the magnitude of acceleration of a tri-axis accelerometer. I want to find a reference time window that I can ...
- 123
5
votes
3
answers
5k
views
How do I compute/estimate the variance of sequential data? [duplicate]
Say I have a (infinite) sequences like 1, 3, 2, 2, 1, 3 ...
I want to estimate their mean and variance of the sequence at time $t$.
But I won't have enough storage to keep all the data seen ...
- 16.8k
3
votes
2
answers
675
views
Detect if an incoming value in streaming data is an outlier
I am reading a sensor that gives data. Sometimes some data is false. I can store some samples before and I would like to detect a glitch on the fly.
Process :
Values are integers (distances in ...
- 133
1
vote
1
answer
163
views
Finding statistically significant "Outliers" from sequential data
I have a need to find data points whose values are statistically different (significant) from sequential data points. For example I'm looking at weekly data points and as new data points are added I ...
- 167
0
votes
0
answers
38
views
What are some major theories on picking the right number for the sample window size in time series analysis?
for example the number of samples to run the moving average, or the number of samples for sequential hypothesis testing. Or if there is a control scheme going on what is the best time window for an ...
- 167
4
votes
1
answer
1k
views
Basic questions about stochastic gradient descent / Robbins and Monro algorithm
I have a LOT of time series observations and I would like to estimate a simple AR(1) model
$$
y_t =c+ \phi y_{t-1}+ \varepsilon_t \qquad \varepsilon_t \sim \text{N}(0, \sigma^{2})
$$
with parameters ...
- 41
3
votes
1
answer
87
views
Correlation on ordered subset
Imagine a hypothetical scenario in which a ball is thrown along a straight line. During flight, the position is continually sampled; however, at some distance, the sampling fails and only noise is ...
- 55
1
vote
2
answers
342
views
Sampling from a dynamic population
I need to create a sample of a given size from a population. However, the population is dynamic, that is, comes as a stream of items, and every item has a "time stamp" based on its location in this ...
- 569
5
votes
0
answers
234
views
Detecting trends in a data stream in real-time
I'm trying to detect trending topics on Twitter in real-time. What I'm doing is every time I get a tweet I assign the tweet to the cluster that talks about the same topic as the tweet. Regardless of ...
- 8,411
12
votes
2
answers
10k
views
Time series forecasting lookback windows -- sliding or growing?
Are there any good reasons to prefer a sliding model training window to a growing window in online time series forecasting (or vice versa)? I'm particularly referring to financial time series.
I ...
- 3,886
5
votes
0
answers
532
views
Reference for implementing generalized likelihood ratio test to determine online whether time-series mean has shifted
What is a reference that describes the "generalized likelihood ratio" test to determine online (i.e., meaning that we add an observation, then check, then add an observation, then check) whether the ...
0
votes
2
answers
191
views
How do I model time to an event with online data?
I am looking at streaming data (i.e. online model), and looking for a specific discrete event. I want to stochastically model the time until this even happens, or if easier, say, model the probability ...
- 31
10
votes
4
answers
3k
views
How to handle online time series forecast?
I have been dealing with the following problem. I have kind of a real time system and every time frame I read its current value, creating a time series (such as 1, 12, 2, 3, 5, 9, 1, ...). I'd like to ...
- 101
2
votes
1
answer
7k
views
Efficient method/technique to update covariance matrix
A covariance matrix of multivariate random variable can be constructed given a time-series random variables.
Eg. If you observe a student's performance in different objects (Math, English, Physics, ...
- 283
5
votes
2
answers
2k
views
Online method for detecting wave amplitude
I would like to measure the amplitude of waves in a noisy time-series on-line. I have a time-series that models a noisy wave function, that undergoes shifts in amplitude. Say, for example, something ...
- 4,997