Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [real-time]

The tag has no usage guidance.

1
vote
0answers
9 views

Nested CV with Online Learning

I have a time series binary classification dataset. I am implementing an online learning Logistic Regression algorithm in Sklearn and am cross validating with Sklearn's TimeSeriesSplit method. I am ...
0
votes
0answers
8 views

Marketing Funnel Percentage Calculations

I implement a marketing funnel in python and I have an issue around the topic that I can't solve neither in statistical level. Lets say that we have the funnel A which contains ...
1
vote
0answers
14 views

Correlation between “ Time of the day” and an ordinal variable

My hypothesis is The compliance score (Dependent/ ordinal variable) varies during a different time of the day. for example, I would expect high compliance between regular working hours 7 am - 5 pm -...
-2
votes
1answer
46 views

Calculate score for multiple variables [closed]

My question is similar to this. I have a large geographical area divided into small cells (approximately 2.25 km$^2$). I'm calculating some value (heat) for individual cells dynamically. It is an ...
0
votes
0answers
15 views

model selection for peak findings

I need to create a model, that finds when there will be a significant chance of occurring High peak in the signal. My data has N length of inputs and corresponding N length of outputs. For example, ...
2
votes
2answers
397 views

How to frequently update classification model with new training data?

How do I incorporate a new stream of data into my classification model? Do I have to retrain the model from the beginning every time I want to incorporate new data, or can I update the existing model ...
1
vote
1answer
73 views

Real Time Recurrent Learning (RTRL) Implementations in R

Are there any recommended RTRL packages for R? I'm specifically looking for online LSTEM or GRU neural network implementations. Thank you!
0
votes
1answer
43 views

Which data needs to be collected and what to do with it. I need step by step process with explanation for understanding.

I sell a given product through 1000 shops & average revenue for that product is 2400 dollars per week per shop, with a standard deviation of 3100 dollars. I want to analyze, the offer ' three for ...
-1
votes
1answer
41 views

Sample size for a research paper (for Mean+SD type of data)

In my research paper I proposed a software and have experiment it on 72 users. But the reviewer has objected on the sample size and asked to justify it or increase it. below is the results of our ...
0
votes
0answers
21 views

Looking for a Classifier dealing with Real Time input

I need some suggestions on 'what classifiers I should use' for the problem I am going to describe below. I have a sensor that is feeding vectors at a rate of 100Hz. The vectors are 750 in length. ...
0
votes
2answers
49 views

Clustering with Self Organizing Maps including time, date and month as attributes

I am about to start up a project on pattern recognition in a highdimensional dataset holding information on transactional salesdata for a company. In that manner I have decided to use the method of ...
1
vote
1answer
212 views

Algorithms for real-time classification of segments in noisy time-series data

I’m trying to detect various features of a toy train track while driving on it: The primary input is data from an optical sensor. The following image shows the recorded signal when driving over the ...
0
votes
1answer
78 views

Choosing best time series model at run time for multiple time series

I am using methods from R forecast package to forecast revenue for 20,000 sectors. How to choose best time series model for each sector at run time?
4
votes
5answers
2k views

is K-Means clustering suited to real time applications?

I want to segment a sequence of RGB images (basically it's a video) based on their colors in real time. KMeans is an easy and intuitive algorithm to use in this case, but it's execution time is very ...
0
votes
0answers
20 views

How can we inject external background knowledge into data stream mining algorithm?

If we want to include the context as a background knowledge, how can we do this when using a data stream mining algorithm. What are the possible forms of the background knowledge and how can they be ...
0
votes
1answer
322 views

Using time stamps as input to Neural Networks to reconstruct particle tracks

So here's the thing: consider a detector, made of several vertical strings, each hosting 60 sensors distributed equidistantly, as shown in the following picture (dark dots are sensors): Particles ...
0
votes
1answer
899 views

Difference between 'Time domain features' and 'frequency domain features'?

I have a time series data of accelerometer in X,Y,Z axis. Data is not sampled at a constant sampling rate(but is close to 100 Hz). In the paper I am referring, it mentions that for feature selection I ...
1
vote
2answers
69 views

Maximizing car park usage efficiency

I'm trying to create a predictive model for a closed car park. This model is expected to estimate available parking spaces given date and time. Brief information: the car park has a limited capacity ...
0
votes
0answers
62 views

How do I calculate the running variance? [duplicate]

I only get one value at a time in the system that I am developing. However whenever I get my first value I should already be able to calculate a variance. Every time I get a new value I should ...
1
vote
1answer
39 views

A way to estimate if a random variable has shifted beyond a threshold

I have a random variable estimated over time by an online algorithm. I have the mean and variance of the random Gaussian variable at every step t. I expect the time series to have sudden shifts. What ...
2
votes
0answers
192 views

Real time prediction system with R [closed]

I am trying to build a real-time prediction system for my application. I am yet to actually build any system involving R. I have some data on MySQL which I'm using to train the model, and I want to ...
1
vote
0answers
48 views

Time series data with a binary output, current state is correlated with past few states if current “unknown” state is 1

Background: I am performing a pattern matching on some data which is being generated real time. I perform the pattern matching on this newly generated data at every 1 second. Every time a match is ...
6
votes
0answers
173 views

Testing for Benford Law in real time

Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution ...
0
votes
0answers
1k views

Real world practical Outliers/Anomalies

I want to have some real life and practical examples for anomalies that can be expressed like a scenario. For example in a country following left-hand driving, a vehicle moving in a wrong direction ...
0
votes
1answer
379 views

local linear regression with streaming data

From a data stream i'm receiving a pair of measurements consisting of a current consumption and a current percentage every second. By accumulating the consumption over time it will represent ...
2
votes
1answer
232 views

Classification in real time without prior knowledge of the number of classes

Is there an implemented algorithm (with python/R or java in preference) that can classify incoming data from an unknown generator with absolutely no prior knowledge or assumption. For example: Let G ...
7
votes
1answer
5k views

Machine learning for pattern recognition in realtime sensor data

I'm working on a project where we need to detect patterns in a sensor's output to find out if a given event occurred. Given my limited experience with machine learning, I was wondering if someone ...
3
votes
1answer
7k views

Real time updating of training data and classification model

Setup: I have a couple of binary classification models based on Logistic Regression and Gradient Boosted Trees. Currently I train the model offline and use it to predict the class of incoming data. ...
2
votes
0answers
130 views

False Positives in Real Time Classification

I am doing a sliding window binary classification. I have time series data and I am running a time window over this data and let a classifier produce a decision probability. Based on this probability, ...
1
vote
0answers
35 views

Detecting mean difference between two observed stochastic processes

Suppose to collect real time two set of samples coming from two sources so defined: S1: gaussian(m1, s1) S2: gaussian(m1+m2, s1+s2) with probability p S2: gaussian(m1, s1) with probability 1-...
1
vote
1answer
277 views

Random Forest Algorithm Where to start?

I need to solve a real time classification problem using random forest algorithm.Can anyone suggest any book,website where I should start looking it ?
0
votes
1answer
170 views

Time series tracking queue optimization problem

In order to track prices of many different products from different sources, I must optimally schedule a group of trackers dedicated to price collection (ie. collect one price at a time for each ...
2
votes
0answers
1k views

Detecting a trend to increase, in a time series, in real time

Probably, someone who's into technical analysis of share prices eats stuff like this for breakfast. Me, I couldn't devise a theoretically acceptable approach. I have this thing (private working set, ...
0
votes
1answer
92 views

What is the seasonality of a data stream and how to use it to estimate the stream attributes?

I am trying to understand what is the rigorous definition of seasonality is, but couldn't find it. Is it a mathematical tool at all? I need to analyze a stream coming from one source and be able to ...
0
votes
1answer
91 views

Which clustering algorithm shall I use?

I need some help My project aims to develop algorithms for spatial temporal analysis of Flickr, Twitter and Foursquare databases to detect any kind of significant changes, named as “Event” in real ...
0
votes
1answer
70 views

Find sudden change or high variation in a dynamic real-time stream of real continuous number

I have a stream of real continuous numbers (Figure. 1). The stream flows from left to right. At $t_1$ the first number entered the stream. Every time a new number enters the stream I check if there is ...
0
votes
0answers
262 views

Entropy as measure of order in data streams

Is it sound/allowed to use entropy as a measure of (non-) uniformity of a data stream? E.g. I calculate the Shannon entropy with the standard formula based on various measures in the data stream. ...
1
vote
1answer
272 views

How to quickly collect data (survey) from students in class?

What would be an easy and simple way to get online responses from students in class for use in class demonstrations. E.g. Height and Weight. A quick survey method in a computer class could make lab ...
0
votes
0answers
81 views

Detecting a burst in real-time?

I'm tracking the value of a variable over time. I would like to detect if the value of the variable is bursting (having an unusual increase in it's value) in real-time. How can I do that?
12
votes
1answer
3k views

Algorithm for real-time normalization of time-series data?

I'm working on an algorithm that takes in a vector of the most recent data point from a number of sensor streams and compares the euclidean distance to previous vectors. The problem is that the ...
27
votes
4answers
43k views

Simple way to algorithmically identify a spike in recorded errors

We need an early warning system. I am dealing with a server that is known to have performance issues under load. Errors are recorded in a database along with a timestamp. There are some manual ...
3
votes
1answer
661 views

Realtime data change detection

I have been asked to do some real time data analysis. The data values represent parameters of phone calls for a telco (like number of calls, call length, etc.) If the numbers suddenly drop or spike, ...
49
votes
6answers
13k views

Efficient online linear regression

I'm analysing some data where I would like to perform ordinary linear regression, however this is not possible as I am dealing with an on-line setting with a continuous stream of input data (which ...
3
votes
2answers
1k views

Graphing real-time data from a text file

I have a process which writes statistics from a server system to a file each second in this format: label1 label2 label3 344 666 787 344 849 344 939 994 344 ...
50
votes
7answers
26k views

Period detection of a generic time series

This post is the continuation of another post related to a generic method for outlier detection in time series. Basically, at this point I'm interested in a robust way to discover the periodicity/...
84
votes
14answers
62k views

Simple algorithm for online outlier detection of a generic time series

I am working with a large amount of time series. These time series are basically network measurements coming every 10 minutes, and some of them are periodic (i.e. the bandwidth), while some other aren'...
7
votes
6answers
2k views

Modeling of real-time streaming data?

I am interested in tools/techniques that can be used for analysis of streaming data in "real-time"*, where latency is an issue. The most common example of this is probably price data from a financial ...