Questions tagged [sequential-analysis]
Hypothesis testing without a fixed sample size.
0
votes
0answers
12 views
Representative Pattern Extraction from Time Series using LSTM
I am interested in extracting a representative pattern from time series having variable time periods. I have attached an image for reference. I would like to know if a LSTM would be the appropriate ...
0
votes
0answers
24 views
Is there a name for this? Updating class probabilities, online
An object is of an unknown class $y$. We receive a stream of measurements $x_1, x_2, ...$ of the object. Every time we receive a new measurement, we want to update our estimate of the class label $y$ (...
1
vote
0answers
10 views
Average Sample Number using Page's CUSUM procedure?
I am trying reproduce the Table 1 results from the page 12 using the equation given the page 13.To access the journal article please click https://arxiv.org/pdf/math/0605322.pdf. The corresponding ...
2
votes
1answer
15 views
Does an interim sample size re-estimation increase type 1 error if based on the overall event rate?
Suppose that a sample size is to be calculated for a trial: a presumed effect size summarizes the effect of intervention (relative risk RR $\exp(\theta_1)$), and background gives the rate of events in ...
0
votes
0answers
13 views
Sequence Prediction with noise / gap using markov models
I'm trying to understand if Markov models can account for a "noise event" when predicting the next item in a sequence.
For instance, if i have very frequently occurring (noise) event "F", can a ...
2
votes
0answers
28 views
Mixed-effects modelling in a sequential analysis design
I'm planning a study and am considering going with a sequential analysis design (as described here: https://psyarxiv.com/9yegd/download?format=pdf). This design allows one to take multiple looks at ...
0
votes
0answers
19 views
How to methodologically show that a given 'time-series/sequential' data is not really sequential?
I have an apparent time-series/sequencial (supervised: multi-class classification problem) dataset with each data-point time-stamped. However, some domain intuition tells me that the data is static ...
-1
votes
1answer
27 views
Reverse the shuffling phase
It is very convenient to have sklearn library to split and shuffle the data with
...
0
votes
0answers
32 views
How to encode text and categorical variables together?
I have two groups of texts that are very similar (e.g. reviews written on fridays and reviews written on mondays), and I want to build a LSTM that can classify them into positive and negative reviews. ...
4
votes
1answer
74 views
Can the Bayesian but not the frequentist “just add more observations”?
Since the frequentist's p-values are uniformly distributed under the null hypothesis, it is a highly problematic practice to add more and more data to your sample until you find a significant result. ...
1
vote
0answers
12 views
Sequential mining technique for categorical events?
I'm facing a problem where I have a short (2-8) sequence of discrete events, of variable length, and a final state. Ex.: cold, warm, warm -> soft clothing.
I would like to determine what are the main ...
2
votes
0answers
43 views
Predicting elections with a random sample
If I have a population of exactly 100.000 voters, and I know they all voted in either candidate A, B, C, D or E. (e.g where the results of a random sample of 100 voters are A = 40%, B = 30%, C = 10%, ...
1
vote
0answers
18 views
Applying Sequential Probability Ratio Test (SPRT) on a Vector Signal?
I am trying to apply SPRT on a vector signal (of residual) for detecting faults occurring in the sensors' measurement of an aircraft. The vector signal contains measurements from five different ...
0
votes
0answers
34 views
Is the Wald Binomial Proportion Confidence Interval good for sequential hypothesis testing?
I'm working with a platform that uses the Wald Binomial Proportion Confidence Interval for hypothesis testing (AB testing conversion rates on a landing page). Info on methodology: https://support....
5
votes
1answer
165 views
Stopping rules in group sequential designs
I have read that in group sequential designs, having a maximum of $k$ stops, for the $j$-th stop $j < k$ there are 4 bounds chosen $a_j,b_j, c_j, d_j$ such that the three decisions are considered. ...
1
vote
0answers
21 views
power calculation for two stage binomial type model
This is a power-type calculation for a Bernoulli/binomial question in two stages.
Suppose you are planning an experiment which starts with a test for an event on $N$ experimental units. The event ...
12
votes
5answers
1k views
Determining sample size before starting an experiment or run the experiment indefinitely?
I studied statistics years ago and have forgotten it all so these may seem like general conceptual questions than anything specific but here is my issue.
I work for an e-commerce website as a UX ...
1
vote
0answers
21 views
How many samples do I need to estimate a gaussian distribution?
Background: I'm comparing the performance of two algorithms. The data is noisy so the results tend to form a nice normal distribution.
I'd like to estimate the mean with a high confidence: up to 1% ...
0
votes
0answers
17 views
In expectation, sequential probability ratio testing requires larger sample than classical testing?
Sequential probability ratio test involves a stopping time defined as $T = \min\left\{n\ge 1: l_n\notin \left(A,B\right)\right\}$ where $A,B$ are some constants such that $0<A<B<\infty$. In ...
3
votes
0answers
23 views
Advice for conducting the same experiment several times?
I'm very new to Bayesian statistics, but I've found myself in a situation where they might be able to help. I'll be running the same experiment repeatedly and analyzing the data every few months. The ...
1
vote
0answers
132 views
Understanding interim analysis and alpha spending functions
I have to perform an A/B test with interim analysis and found some literature about methods designed to preserve the global Type I error rate by adjusting the $\alpha$ level for each peeking.
...
1
vote
0answers
37 views
Convergence of a sequence in finite number of steps
Here is the setup of my problem. It is a sequential problem and there are two possible actions A and B. Now, when either action $A$ or $B$ is taken at the $j$th time point, we observe some outcome say ...
2
votes
0answers
37 views
Optimal denoising of a few features for latent variable modeling
I am fitting latent variable models to noisy, high-dimensional data. The dimension is 15,000 to 20,000; there are 2,000 observations. There is a certain latent structure that I observe -- essentially, ...
0
votes
1answer
303 views
Bayesian updating with discrete priors + possibly unknown classes
I'm following along with some lecture notes on Bayesian updating with discrete priors. They give an example problem to illustrate some of these concepts, which I briefly restate here:
Someone tells ...
1
vote
1answer
83 views
sample size calculation for one-sample log rank test with interims (ie group sequential)
I wonder if there is any advice for performing a sample size calculation for survival data using the one-sample log rank test with interims (ie group sequential) - ie with O'Brien-Fleming or some ...
0
votes
1answer
35 views
Finding expected value with $3$ unknown variables
Question:
There are n marbles in a bag, player $1$ picks out $x$ marbles where $x \leq n$, marks them, and puts them back. Player $2$ picks out $y$ marbles where $y \leq n$. What is the expected ...
2
votes
1answer
368 views
Complexities of interpreting A/B testing power & significance calculations
Apologies up front for not being a statistician. I've been searching high and low and I cannot find a sufficiently detailed explanation how to interpret differences in A/B testing power and ...
2
votes
0answers
26 views
Just how bad is it to determine when to stop data collection based on the resulting CI?
Say I want to be $95\%$ sure of the mean $\bar X$, with a CI of size $2$ ($1$ in each direction).
The following is bad practice:
collect data until $1.96\frac{s}{\sqrt{n}}=1$.
What is the extent ...
2
votes
0answers
122 views
Building RNNs on mixed sequential and non-sequential data
I have a data set that is a bunch of windows, for each window I want to perform regression. The windows themselves have four sequential features that span back about 24 time steps. Additionally each ...
-1
votes
2answers
63 views
substring clustering with noise
I have sequence which is a repetition of a few strings. I have identified the start and end markers of these strings in the sequence and now, I want to cluster similar strings together. The problem is ...
0
votes
0answers
17 views
Derive sequencing probabilities from univariate data
Given a variable measured sequentially e.g. -6,-3,0,2,-3,1,-5,7,-47.. etc, what are some methods to derive some reliable sequencing probabilities?
e.g. if i-2>0 and i-1 >0 then i>0 etc.
0
votes
1answer
53 views
How to compute expected value?
Hi I am trying to compute this:
It's a denominator of ANS function in sequential statistics.The result is:
I set "sigma1"=1 and "mi" is considered as a constant.Do you have any idea how to get the ...
1
vote
1answer
68 views
Determining the number of days to reach significance
Apologies for what I think will be a simple question - it has been some years since I've take statistics.
I run a business and we are attempting to determine the number of days required to reach a ...
2
votes
0answers
2k views
How to use LSTM/GRU/RNN for sequence classification with Tensorflow/keras? [closed]
I want to use LSTMs and RNNs for sequence classification problem. I have features extracted from video segment of 10 sec @ 30 fps i.e. total of 300 frames. For each frame I have a feature ...
0
votes
1answer
102 views
How can I make a mean sequential estimation?
I am an agronomist trying to estimate the average weight of fruit in a orchard. My first option is to grab 1000 fruit and get their weight mean. But as you can see, if i can get this with less fruits, ...
0
votes
1answer
32 views
Group sequential method subjecting to non-normality
In the context of group sequential methods(GSM) for a clinical trial with two treatment arms, it is usually assumed that data follows normal distribution $N(\mu,\sigma^2)$. Then GSM such as Pocock's ...
1
vote
0answers
26 views
maximum cumulative decision limit
It is well-known that it is problematic to stop an A/B test before we obtain a required number of samples. This website "A/B Testing Rigorously (without losing your job)" explains how to stop safely ...
1
vote
1answer
173 views
How can I solve the sequence prediction problem?
Let's say I'm given a stream of characters, one at a time. Some example strings of characters that could occur might be "01234" or "hello how are". They might even be encodings of waveforms as ...
8
votes
3answers
382 views
High autocorrelation when taking the L-th order of difference of a sequence of independent random numbers
To explain this question in more detail, I'll first elaborate my approach:
I simulated a sequence of independent random numbers $X = \{x_1,...,x_N\}$.
I then take $L$ times the difference; i.e. I ...
1
vote
0answers
49 views
Terminology for sequential estimation with randomized stopping time
Consider a sequence of i.i.d Bernoulli RV's $\{X_i\}_{i \in \mathbb N}$ with parameter $p$. Based on this sequence we build a sequential estimator of $p$ (using inverse binomial sampling for example). ...
6
votes
0answers
165 views
Testing for Benford Law in real time
Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution ...
2
votes
0answers
46 views
What type of analysis for sequential data can I use to identify patterns in collaborative activities
I have a data set, which is collected every 5 second for a total of one hour using a special type of wearable device. Each data point is the amount of verbal participation ratio (measured as percent) ...
12
votes
0answers
713 views
Bayesian Q-learning
Suppose that, for every state $s$, there is a set of actions $\mathcal{A}(s)$ that can be chosen in that state.
Let $Q(s, a)$ denote the expected utility of choosing action $a \in \mathcal{A}(s)$ in ...
0
votes
0answers
24 views
Is there a statistical correction that works if the number of experiments depends on the result of the experiments?
Suppose that I am running an experiment that gives us some confidence $c$ about a hypothesis.
If I intend to run the experiment $n$ times, I understand there are some ways of choosing a $c'$ so that ...
4
votes
0answers
489 views
Expectation and variance of sample mean with random sample size
I have a question regarding sampling where the sample size itself is a random variable.
Say I have two sub-populations $A$ and $B$ from which I can sample a real valued random variable with ...
0
votes
0answers
65 views
changepoint detection and its analysis
I applied changepoint detection provided by ecp package against a given time series. The time series plot marked with identified change points is shown as follows. ...
2
votes
1answer
702 views
Should one adjust analyses if interim analysis is planned for an outcome other than the primary outcome?
It is widely accepted that analyses of clinical trials have to be adjusted if interim analyses are planned in order to decide if early termination is necessary due to evidence for superiority of one ...
0
votes
1answer
81 views
Sequential parameter estimation of multiple discrete random walks
We have $M$ Bernoulli random walks, $X_1$ has parameter $p< 1/2$ and all the rest have parameter $1/2$. The steps are $\pm 1$. I want to identify the parameter $p$ random walk with probability of ...
1
vote
1answer
809 views
Sequential classification methods
I am curious to know if there are methods that exists for sequential modeling of binary outputs? Let me give an example to help further clarify the question:
I have a problem where I have binary ...
5
votes
0answers
125 views
textbooks/literature/resources for online learning/ time series/ sequential analysis? [closed]
I have read (the contents page of) machine learning introductory textbooks such as Pattern Recognition and Machine Learning, Machine Learning A Probabilistic Perspective, and The Elements of ...
