Questions tagged [real-time]
The real-time tag has no usage guidance.
53
questions
1
vote
1
answer
62
views
Which machine learning methods that leverage historic and real-time data should be considered for timeseries short-term forecasting?
Some clarifications to my question:
The data I have available for use is: (a) historic data of features and ground truth on 60-minute interval, (b) real-time data of features on 60-minute interval, (...
- 111
0
votes
1
answer
66
views
How is the Q value translated to action in DRL framework presented in Gu, Shixiang et al (2016)
I was reading the paper Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates and I kind of understand everything. The results also seem very interesting, enabling ...
- 101
1
vote
0
answers
25
views
Which correlation algorithm or machine learning concept to use for categorizing sensor data?
I am trying to find a method where I can detect patterns from acceleration sensor data on a wheeled mobile robot, ideally using Python. I have limited experience in (supervised) machine learning and ...
- 11
-1
votes
2
answers
39
views
Another term for "real time decisioning"
There is another term for "real time decisioning," (RTD) and I need it to locate a paper (by searching). I've been beating my head over this all day. I'm thinking it's "[something] ...
- 1
0
votes
0
answers
149
views
Real-time anomaly detection for online time -series
I am new to the anomaly detection world and am dealing with a project to detect real-time anomalies for a time-series in a fraud detection schema. I read the answer by Rob Hyndman here and like the ...
- 51
0
votes
1
answer
673
views
Machine learning for time-series forecasting
I am new to machine learning and have been trying to learn. I am forecasting AQI using ExtraTreesRegression model in Python. My training and testing results are coming good. While testing my model my ...
0
votes
0
answers
40
views
Find plausible peaks in streamed data
i have got a signal of a streamed source which produces values like in the picture.
I want to get the "real" peaks (blue circles).
But the noisy peaks (red circles) mess up the peak search.
The ...
- 31
2
votes
1
answer
52
views
Nested CV with Online Learning
I have a time series binary classification dataset. I am implementing an online learning Logistic Regression algorithm in Sklearn and am cross validating with Sklearn's TimeSeriesSplit method.
I am ...
- 674
1
vote
1
answer
65
views
Correlation between " Time of the day" and an ordinal variable
My hypothesis is The compliance score (Dependent/ ordinal variable) varies during a different time of the day.
for example, I would expect high compliance between regular working hours 7 am - 5 pm -...
- 11
-2
votes
1
answer
2k
views
Calculate score for multiple variables [closed]
My question is similar to this.
I have a large geographical area divided
into small cells (approximately 2.25 km$^2$). I'm calculating some value (heat) for individual cells dynamically. It is an ...
- 101
0
votes
0
answers
25
views
model selection for peak findings
I need to create a model, that finds when there will be a significant chance of occurring High peak in the signal. My data has N length of inputs and corresponding N length of outputs. For example,
...
- 101
6
votes
2
answers
4k
views
How to frequently update classification model with new training data?
How do I incorporate a new stream of data into my classification model? Do I have to retrain the model from the beginning every time I want to incorporate new data, or can I update the existing model ...
- 1,030
2
votes
1
answer
452
views
Real Time Recurrent Learning (RTRL) Implementations in R
Are there any recommended RTRL packages for R? I'm specifically looking for online LSTEM or GRU neural network implementations. Thank you!
0
votes
1
answer
53
views
Which data needs to be collected and what to do with it. I need step by step process with explanation for understanding.
I sell a given product through 1000 shops & average revenue for that product is 2400 dollars per week per shop, with a standard deviation of 3100 dollars. I want to analyze, the offer ' three for ...
-1
votes
1
answer
95
views
Sample size for a research paper (for Mean+SD type of data)
In my research paper I proposed a software and have experiment it on 72 users. But the reviewer has objected on the sample size and asked to justify it or increase it. below is the results of our ...
- 1
0
votes
0
answers
46
views
Looking for a Classifier dealing with Real Time input
I need some suggestions on 'what classifiers I should use' for the problem I am going to describe below.
I have a sensor that is feeding vectors at a rate of 100Hz. The vectors are 750 in length. ...
- 1
0
votes
2
answers
119
views
Clustering with Self Organizing Maps including time, date and month as attributes
I am about to start up a project on pattern recognition in a highdimensional dataset holding information on transactional salesdata for a company. In that manner I have decided to use the method of ...
- 133
3
votes
1
answer
557
views
Algorithms for real-time classification of segments in noisy time-series data
I’m trying to detect various features of a toy train track while driving on it:
The primary input is data from an optical sensor. The following image shows the recorded signal when driving over the ...
- 131
1
vote
1
answer
193
views
Choosing best time series model at run time for multiple time series
I am using methods from R forecast package to forecast revenue for 20,000 sectors. How to choose best time series model for each sector at run time?
- 523
4
votes
5
answers
6k
views
is K-Means clustering suited to real time applications?
I want to segment a sequence of RGB images (basically it's a video) based on their colors in real time. KMeans is an easy and intuitive algorithm to use in this case, but it's execution time is very ...
- 149
0
votes
0
answers
26
views
How can we inject external background knowledge into data stream mining algorithm?
If we want to include the context as a background knowledge, how can we do this when using a data stream mining algorithm. What are the possible forms of the background knowledge and how can they be ...
- 1
2
votes
1
answer
993
views
Using time stamps as input to Neural Networks to reconstruct particle tracks
So here's the thing: consider a detector, made of several vertical strings, each hosting 60 sensors distributed equidistantly, as shown in the following picture (dark dots are sensors):
Particles ...
- 133
0
votes
1
answer
1k
views
Difference between 'Time domain features' and 'frequency domain features'?
I have a time series data of accelerometer in X,Y,Z axis. Data is not sampled at a constant sampling rate(but is close to 100 Hz). In the paper I am referring, it mentions that for feature selection I ...
- 213
1
vote
2
answers
81
views
Maximizing car park usage efficiency
I'm trying to create a predictive model for a closed car park. This model is expected to estimate available parking spaces given date and time.
Brief information:
the car park has a limited capacity
...
- 43
0
votes
0
answers
242
views
How do I calculate the running variance? [duplicate]
I only get one value at a time in the system that I am developing. However whenever I get my first value I should already be able to calculate a variance.
Every time I get a new value I should ...
- 101
1
vote
1
answer
56
views
A way to estimate if a random variable has shifted beyond a threshold
I have a random variable estimated over time by an online algorithm. I have the mean and variance of the random Gaussian variable at every step t. I expect the time series to have sudden shifts. What ...
- 11
2
votes
0
answers
242
views
Real time prediction system with R [closed]
I am trying to build a real-time prediction system for my application. I am yet to actually build any system involving R.
I have some data on MySQL which I'm using to train the model, and I want to ...
- 121
1
vote
0
answers
65
views
Time series data with a binary output, current state is correlated with past few states if current "unknown" state is 1
Background: I am performing a pattern matching on some data which is being generated real time. I perform the pattern matching on this newly generated data at every 1 second. Every time a match is ...
- 21
7
votes
1
answer
364
views
Testing for Benford Law in real time
Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution ...
- 171
0
votes
0
answers
4k
views
Real world practical Outliers/Anomalies
I want to have some real life and practical examples for anomalies that can be expressed like a scenario.
For example in a country following left-hand driving, a vehicle moving in a wrong direction ...
- 201
0
votes
1
answer
773
views
local linear regression with streaming data
From a data stream i'm receiving a pair of measurements consisting of a current consumption and a current percentage every second. By accumulating the consumption over time it will represent ...
- 123
2
votes
1
answer
296
views
Classification in real time without prior knowledge of the number of classes
Is there an implemented algorithm (with python/R or java in preference) that can classify incoming data from an unknown generator with absolutely no prior knowledge or assumption.
For example:
Let G ...
- 121
8
votes
1
answer
6k
views
Machine learning for pattern recognition in realtime sensor data
I'm working on a project where we need to detect patterns in a sensor's output to find out if a given event occurred. Given my limited experience with machine learning, I was wondering if someone ...
- 85
3
votes
1
answer
8k
views
Real time updating of training data and classification model
Setup:
I have a couple of binary classification models based on Logistic Regression and Gradient Boosted Trees.
Currently I train the model offline and use it to predict the class of incoming data.
...
- 93
2
votes
0
answers
142
views
False Positives in Real Time Classification
I am doing a sliding window binary classification. I have time series data and I am running a time window over this data and let a classifier produce a decision probability. Based on this probability, ...
- 129
1
vote
0
answers
38
views
Detecting mean difference between two observed stochastic processes
Suppose to collect real time two set of samples coming from two sources so defined:
S1: gaussian(m1, s1)
S2: gaussian(m1+m2, s1+s2) with probability p
S2: gaussian(m1, s1) with probability 1-...
- 111
1
vote
1
answer
299
views
Random Forest Algorithm Where to start?
I need to solve a real time classification problem using random forest algorithm.Can anyone suggest any book,website where I should start looking it ?
- 11
1
vote
1
answer
249
views
Time series tracking queue optimization problem
In order to track prices of many different products from different sources, I must optimally schedule a group of trackers dedicated to price collection (ie. collect one price at a time for each ...
- 111
2
votes
0
answers
1k
views
Detecting a trend to increase, in a time series, in real time
Probably, someone who's into technical analysis of share prices eats stuff like this for breakfast. Me, I couldn't devise a theoretically acceptable approach.
I have this thing (private working set, ...
- 121
0
votes
1
answer
147
views
What is the seasonality of a data stream and how to use it to estimate the stream attributes?
I am trying to understand what is the rigorous definition of seasonality is, but couldn't find it. Is it a mathematical tool at all?
I need to analyze a stream coming from one source and be able to ...
- 739
0
votes
1
answer
104
views
Which clustering algorithm shall I use?
I need some help
My project aims to develop algorithms for spatial temporal analysis of Flickr, Twitter and Foursquare databases to detect any kind of significant changes, named as “Event” in real ...
0
votes
1
answer
94
views
Find sudden change or high variation in a dynamic real-time stream of real continuous number
I have a stream of real continuous numbers (Figure. 1). The stream flows from left to right. At $t_1$ the first number entered the stream. Every time a new number enters the stream I check if there is ...
- 155
0
votes
0
answers
335
views
Entropy as measure of order in data streams [duplicate]
Is it sound/allowed to use entropy as a measure of (non-) uniformity of a data stream?
E.g. I calculate the Shannon entropy with the standard formula based on various measures in the data stream. ...
- 649
2
votes
1
answer
344
views
How to quickly collect data (survey) from students in class?
What would be an easy and simple way to get online responses from students in class for use in class demonstrations. E.g. Height and Weight. A quick survey method in a computer class could make lab ...
- 591
0
votes
0
answers
116
views
Detecting a burst in real-time?
I'm tracking the value of a variable over time. I would like to detect if the value of the variable is bursting (having an unusual increase in it's value) in real-time. How can I do that?
- 8,401
13
votes
1
answer
5k
views
Algorithm for real-time normalization of time-series data?
I'm working on an algorithm that takes in a vector of the most recent data point from a number of sensor streams and compares the euclidean distance to previous vectors. The problem is that the ...
- 250
40
votes
4
answers
63k
views
Simple way to algorithmically identify a spike in recorded errors
We need an early warning system. I am dealing with a server that is known to have performance issues under load. Errors are recorded in a database along with a timestamp. There are some manual ...
- 503
3
votes
2
answers
1k
views
Realtime data change detection
I have been asked to do some real time data analysis. The data values represent parameters of phone calls for a telco (like number of calls, call length, etc.) If the numbers suddenly drop or spike, ...
- 133
67
votes
6
answers
21k
views
Efficient online linear regression
I'm analysing some data where I would like to perform ordinary linear regression, however this is not possible as I am dealing with an on-line setting with a continuous stream of input data (which ...
- 1,015
3
votes
2
answers
2k
views
Graphing real-time data from a text file
I have a process which writes statistics from a server system to a file each second in this format:
label1 label2 label3
344 666 787
344 849 344
939 994 344
...
