Questions tagged [sequential-analysis]
Hypothesis testing without a fixed sample size.
146 questions
1
vote
0
answers
9
views
Sequential probability ratio test for variance
An exercise problem from Hogg and McKean's book "Introduction to Mathematical Statistics" is the following
Exercise 8.4.1. Let $X$ be $N(0,\theta)$ and, in the notation of this section, let $...
0
votes
0
answers
23
views
Term for sampling in groups of a given size
Imagine that you want to inspect members of a population to see if they have a target feature. For example, test people to see if they have a certain disease.
One possible approach is to take a sample ...
- 1,191
12
votes
6
answers
4k
views
Why isn't a confidence level of anything >50% "good enough"?
I took a graduate-level stat course so I feel really dumb asking this, but I'm having trouble with the meaning behind 95% confidence. (Please be nice to me.)
I currently running an A/B test on two ...
- 319
2
votes
0
answers
56
views
How to perform data-driven choice of sample size
Say I want to compare the result of 2 regression model. One is the reference model and the second one is the new model, which I want to know if it performs better. Each model is evaluated with the ...
- 171
1
vote
0
answers
35
views
Fixed-sample size sequential ANOVA or t-test
I have an experiment of fixed sample size where units (20 in each of 2 or 3 arms) are allowed to grow over time. I wish to stop the experiment as soon as there's enough evidence that the arms do not ...
- 1,129
0
votes
0
answers
13
views
Covariance between two standardized statistics - Group sequential designs
I'm working through the book "Group Sequential Methods with Applications to Clinical Trials" by Jennison & Turnbull (1999). I'm particularly interested in understanding the derivation of ...
- 61
0
votes
0
answers
29
views
Sequential Probability Ratio Test of ANOVA
I am running a sequential experiment that consists in 60 units in three treatment arms, so 20 units in each arm. We can assume that in each arm, the values recorded for each unit are normally ...
- 1,129
2
votes
0
answers
20
views
Distribution of tuples in chaotic sequences
I study infinite aperiodic sequences like Thue-Morse.
Simple substitution rules allow you to get even more complex.
I'm interested in the distribution of tuples in such sequences.
For example, in Thue-...
- 253
0
votes
0
answers
31
views
Lower threshold for Sequential Probability Ratio Test on Contingency Table testing
For testing whether a die is fair, we have the log likelihood ratio:
$$
\Lambda = \frac{1}{2} \sum_{i} \mathrm{observed}_i \log\left(\frac{\mathrm{observed}_i}{\mathrm{expected}_i}\right).
$$
Suppose ...
- 15k
2
votes
0
answers
20
views
How to read and use equations for confidence interval from sequential testing
in the book "Group Sequential and Confirmatory Adaptive Designs in Clinical Trials" by Wasmer and Brannath - they explain on p.85 how to calculate a confidence interval.
They consider a ...
- 241
2
votes
0
answers
44
views
Sequential analysis - adjusting effect size
I am aware that sequential testing is risky, but it makes sense for me to do it anyway. So, I would like to read up, to do it properly.
My idea (which is probably not new):
In my reality, the true ...
- 386
2
votes
0
answers
17
views
Looking to extract patterns from sequences of codes
I have the following problem: I have a registration of people who enter a building, I have the name, entry date and end date. I also have the times at which events occur inside the building.
I want to ...
- 205
3
votes
0
answers
38
views
Techniques for Stopping a Diagnostic Study for Safety
I'm working with a group of physicians who are interested in evaluating a new diagnostic tool for detecting colon cancer in older patients. The gold standard to detect colon cancer is with a ...
- 39.6k
5
votes
1
answer
79
views
How to pick the winner in the "Play the Winner" treatment assignment scheme (Urn model)
"Play the winner" is an intutitive procedure whereby successful treatments in earlier trials have a higher probability of being assigned in later trials. Quoting https://online.stat.psu.edu/...
- 5,447
2
votes
1
answer
369
views
Sample size for 2-arm 2-stage trial: R package Trialsize vs rpact vs gsDesign
Which sample size calculation is correct?
Intended design:
Superiority two-arm trial between drug A (intervention) vs drug B (comparator),
outcome is binomial (response or no response to treatment),
...
- 165
3
votes
0
answers
53
views
prior distribution for iid gaussian, with a known variance
I have been reading Pattern Recognition and Machine Learning by Bishop, and I have a question regarding the prior distribution of an iid Gaussian with known variance.
The relationship $\dfrac{n}{\...
- 9,317
0
votes
0
answers
44
views
PRML: Why does the sequential estimation algorithm of normal parameters use $X_N$ but $X_{n-1}$?
I'm reading sequential estimation section 2.3.5 PRML where Bishop introduces Robbins-Monro algorithm to calculate the root of $f(\theta) = E z | \theta = \int zp(z|\theta) dz, (z, \theta) \sim p(z, \...
- 43
3
votes
1
answer
60
views
Can't reproduce closed form expression from sequential testing
I've been trying to implement and extend some results from the papers "Always Valid Inference" and "Peeking at A/B Tests". The authors provide a closed form expression of the "...
- 41
1
vote
0
answers
49
views
Identifying synchrony across individual time series
We are working with some time series data that look like the following:
We have three individuals (a1,a2,a3) and then several observations at regular time-intervals. Here, we have 12 time intervals. ...
- 379
2
votes
0
answers
88
views
Understanding the Sequential Probability Ratio Test
I am trying to understand how to apply SPRT to the following case.
Random variable $X$ is exponentially distributed according to $f(x, \theta)$. Test the following hypotheses
$$
H_0 : \theta \ge \...
- 121
4
votes
2
answers
148
views
How are group sequential analysis, random walks, and Brownian motion related?
Assume that I am planning a clinical trial comparing two groups using a binary outcome. I will do the $\chi^2$ test after 3 equal enrollment intervals: interim test #1 after $m_1$ enrollments in ...
- 426
0
votes
1
answer
108
views
Can you use Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling?
I have a question regarding the use of Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling.
I am trying to perform Sequential Linear Modeling (with a binary outcome) ...
- 11
4
votes
1
answer
99
views
Is the given procedure SPRT?
In this question, I tried using the fundamental property of the Wald Sequential Probability Ratio Test (SPRT): $P( N = \infty )=0.$
For this I calculated $p_1 = 1- P(-1.5 < S_1 < 2.5 )$, that ...
- 387
0
votes
0
answers
48
views
variance of the sprt stopping time
Is it possible to reduce the variance of the average stopping time of a fixed boundary SPRT by allowing variable boundaries as the test progresses at the price of increasing the average stopping time? ...
- 101
0
votes
0
answers
15
views
Theory behind this natural sequential estimator of the mean
Suppose we have i.i.d. real random variables $X_1, \cdots$. We want to estimate the expectation $E[X_1]$ to within some desired error $\epsilon$, but we do not know the variance. We want to use as few ...
- 145
1
vote
2
answers
1k
views
Sequential Bayesian updating of mean and variance of normal distribution
I am trying to write some code to learn the parameters of a normal distribution. I am new to this, and I have patched together the equations from various sources, which may be part of the problem. In ...
- 23
3
votes
1
answer
93
views
How to parametrize a beta prior for a sequential Bayesian analysis of a proportion?
I'm developing a decision rule for a study using an interim analysis with a Bayesian characterization of the event probability. At a certain point, we will perform a planned stop, and only proceed ...
- 64.8k
0
votes
0
answers
94
views
Does magnitude of difference between two variables affect the confidence level?
I have two sets of data that I'm using proprietary software (TubeBuddy) to do data analysis to find which image is "better." I'm assuming the software is doing a two-sample t-test.
TubeBuddy'...
- 319
2
votes
1
answer
184
views
Distribution for Fraction of Success in a Binomial Setting
So the actual original question I am trying to solve is a little bit different than the title:
In a binomial setting with probability of success $p$, I keep examining observations until a fraction of ...
- 187
1
vote
0
answers
32
views
Should SPRT use explicit hypotheses or reflect observed differences in the data?
Sequential Probability Ratio Testing (SPRT) compares the log-likelihood ratio of the observed data under the null and alternative hypotheses to thresholds based on desired alpha and beta levels.
I am ...
- 158
1
vote
0
answers
67
views
Adjusting effect sizes, p values, and CIs for sequential analysis via simulation
It's fairly easy to write a simulation procedure to calculate, for any given global type 1 error rate, adjusted local "stopping" alphas for interim "looks" in a sequential design, ...
- 234
2
votes
0
answers
356
views
How is the O'Brien-Fleming constant calculated and how does the procedure control error rates?
O'Brien-Fleming is a commonly used interim analysis procedure in clinical trials. The procedure is used when $N$ tests are performed in sequence. For each test $n \in \{1, ..., N\}$, a test statistic $...
- 41
1
vote
1
answer
28
views
Estimating population means after a selection procedure
The problem is somewhat related to sequential procedures (e.g. Paulson; Gupta and Miescke)
I have a hundred engines generating normal random numbers with true (unknown) means $E_1,E_2,E_3...E_{100}$, ...
- 13
1
vote
0
answers
25
views
When can I stop drawing costly samples?
I'm running (costly) experiments and I need to decide when to stop gathering samples. Is the following approach correct?
I'm testing a software package for performance: I measure the time it takes for ...
- 111
0
votes
0
answers
45
views
I do not understand, are Neyman-Pearson rule and Sequential Probability Ratio Test refers to the same thing?
In Bayesian statistics, I remember to Neyman Pearson rule, in which, based on a threshold value, we have the option of reject to make a decision: decision option.
Now, I wonder if it is the same as ...
- 243
1
vote
1
answer
337
views
How to prove SPRT terminates with probability unity?
Can anyone provide me the easily explainable proof of 'SPRT terminates with probability unity'?
To be more specific- 'Prove that SPRT terminates with probability unity'
2
votes
1
answer
72
views
How to explain these jump-discontinuities in the expected stopping time of a SPRT?
Suppose a sequential probability ratio test (SPRT) with $$\alpha=P(\text{Type I error})=P(\text{Type II error})=\beta = 10\%$$ is used to decide between $$\ H_0: p=1/2\quad\text{vs.}\quad H_1: p=b\,$$ ...
- 301
2
votes
1
answer
144
views
Changing sample size during experiment? [duplicate]
Will the significance and power levels hold constant if I don't see statistically significant results and decide to wait to allow for more samples to be collected?
anon
0
votes
2
answers
72
views
When should a problem be approached as a sequential one?
I am working on a problem with a dataset of the form $(x_j,t_i,y_{i,j} )_{(j,i) \in J \times I}$ where $y_{i,j}$ is the output label. Think of progression of illness over time or weather conditions ...
- 102
0
votes
0
answers
101
views
Bayesian sequential updating with current Bayesian sampling software?
I'm having a hard time implementing sequential updating with current software, I don't even know how to start. Basically, I'm having to refit the whole model by simply appending the new data to the ...
user314217
4
votes
0
answers
85
views
Sequential hypothesis test for uniform distribution
Suppose I have a sequence of data points $x_1,x_2,\dots$. I want to do a sequential hypothesis test, were the null hypothesis is that each $x_i$ is iid uniform on $[0,1]$, and the alternative ...
- 6,738
1
vote
0
answers
59
views
Sequential Testing of Poisson Process
My question is related to the paper "Sequential Testing for Poisson Processes" by Peskir and Shiryaev, available here.
Specifically, suppose that there are two states of the world, G(ood) ...
- 11
0
votes
1
answer
466
views
How can I adjust the p-value calculation for an A/B test that is currently running?
I am running an A/B test. I have about 95k samples per variation and 1.29% relative improvement. The test is to see whether the variant converts users to booking better than the control. Test is still ...
- 43
0
votes
1
answer
455
views
Group Sequential Clinical Trial Design - Non-binding efficacy boundaries?
I have been reading up on group sequential clinical trial designs and came across the concept of non-binding futility boundaries. Intuitively, it makes sense to me that continuing the trial after ...
- 57
5
votes
2
answers
777
views
Sequential updating of a Bayesian regression model
I'd like to perform Bayesian regression of $Y$ on $X$. i.e. estimate a posterior $p\left(\theta\mid\mathcal{D}\right)$ of some regression parameter $\theta$ given a dataset $\mathcal{D}$ of pairs of ...
- 451
8
votes
1
answer
599
views
How to distinguish between two biased coins
I have two very biased coins:
The first coin (C1) lands on heads 95% of the time.
The second coin (C2) lands on heads only 1% of the time.
One of the coins has been selected and I want to determine ...
- 105
3
votes
1
answer
704
views
Term for when hypothesis testing is conducted repeatedly until an interesting result is found?
What is the term for when, through accident or intent, many hypothesis tests are conducted until an 'interesting' result is found? That is, the experiment (or variations on it) are repeated until a ...
- 303
1
vote
0
answers
434
views
In R, why do the p-values from anova() change when you add more predictors?
When conducting linear regression in R, I am trying to understand how certain p-values are calculated and what they represent. So far this is my understanding:
The p-values from summary() correspond ...
- 11
6
votes
1
answer
276
views
Equivalence of Contextual Bandit formulations
I find two different types of formulations for Contextual Bandits:
Definition 1: In a contextual bandits problem, there is a distribution $P$ over $(x,r_1,...,r_k)$, where $x$ is the context, $a \in [...
- 896
1
vote
2
answers
335
views
Is sequential probability ratio test bayesian or frequentist or both?
Is sequential probability ratio test bayesian or frequentist or both?
Is there a way to implement a bayes decision rule with "sprt"?
user271077