Skip to main content

Questions tagged [sequential-analysis]

Hypothesis testing without a fixed sample size.

Filter by
Sorted by
Tagged with
2 votes
0 answers
19 views

Distribution of tuples in chaotic sequences

I study infinite aperiodic sequences like Thue-Morse. Simple substitution rules allow you to get even more complex. I'm interested in the distribution of tuples in such sequences. For example, in Thue-...
lesobrod's user avatar
  • 253
0 votes
0 answers
24 views

Lower threshold for Sequential Probability Ratio Test on Contingency Table testing

For testing whether a die is fair, we have the log likelihood ratio: $$ \Lambda = \frac{1}{2} \sum_{i} \mathrm{observed}_i \log\left(\frac{\mathrm{observed}_i}{\mathrm{expected}_i}\right). $$ Suppose ...
shabbychef's user avatar
  • 14.9k
2 votes
0 answers
18 views

How to read and use equations for confidence interval from sequential testing

in the book "Group Sequential and Confirmatory Adaptive Designs in Clinical Trials" by Wasmer and Brannath - they explain on p.85 how to calculate a confidence interval. They consider a ...
clog14's user avatar
  • 231
2 votes
0 answers
39 views

Sequential analysis - adjusting effect size

I am aware that sequential testing is risky, but it makes sense for me to do it anyway. So, I would like to read up, to do it properly. My idea (which is probably not new): In my reality, the true ...
W_vH's user avatar
  • 339
2 votes
0 answers
16 views

Looking to extract patterns from sequences of codes

I have the following problem: I have a registration of people who enter a building, I have the name, entry date and end date. I also have the times at which events occur inside the building. I want to ...
slow_learner's user avatar
3 votes
0 answers
36 views

Techniques for Stopping a Diagnostic Study for Safety

I'm working with a group of physicians who are interested in evaluating a new diagnostic tool for detecting colon cancer in older patients. The gold standard to detect colon cancer is with a ...
Demetri Pananos's user avatar
5 votes
1 answer
71 views

How to pick the winner in the "Play the Winner" treatment assignment scheme (Urn model)

"Play the winner" is an intutitive procedure whereby successful treatments in earlier trials have a higher probability of being assigned in later trials. Quoting https://online.stat.psu.edu/...
Ben Ogorek's user avatar
  • 5,417
2 votes
1 answer
281 views

Sample size for 2-arm 2-stage trial: R package Trialsize vs rpact vs gsDesign

Which sample size calculation is correct? Intended design: Superiority two-arm trial between drug A (intervention) vs drug B (comparator), outcome is binomial (response or no response to treatment), ...
saifulsafuan's user avatar
3 votes
0 answers
39 views

prior distribution for iid gaussian, with a known variance

I have been reading Pattern Recognition and Machine Learning by Bishop, and I have a question regarding the prior distribution of an iid Gaussian with known variance. The relationship $\dfrac{n}{\...
cgo's user avatar
  • 9,217
0 votes
0 answers
42 views

PRML: Why does the sequential estimation algorithm of normal parameters use $X_N$ but $X_{n-1}$?

I'm reading sequential estimation section 2.3.5 PRML where Bishop introduces Robbins-Monro algorithm to calculate the root of $f(\theta) = E z | \theta = \int zp(z|\theta) dz, (z, \theta) \sim p(z, \...
Chia's user avatar
  • 43
3 votes
1 answer
60 views

Can't reproduce closed form expression from sequential testing

I've been trying to implement and extend some results from the papers "Always Valid Inference" and "Peeking at A/B Tests". The authors provide a closed form expression of the "...
user2844936's user avatar
1 vote
0 answers
45 views

Identifying synchrony across individual time series

We are working with some time series data that look like the following: We have three individuals (a1,a2,a3) and then several observations at regular time-intervals. Here, we have 12 time intervals. ...
jalapic's user avatar
  • 379
1 vote
0 answers
79 views

Understanding the Sequential Probability Ratio Test

I am trying to understand how to apply SPRT to the following case. Random variable $X$ is exponentially distributed according to $f(x, \theta)$. Test the following hypotheses $$ H_0 : \theta \ge \...
chuck1's user avatar
  • 111
4 votes
2 answers
143 views

How are group sequential analysis, random walks, and Brownian motion related?

Assume that I am planning a clinical trial comparing two groups using a binary outcome. I will do the $\chi^2$ test after 3 equal enrollment intervals: interim test #1 after $m_1$ enrollments in ...
Mkanders's user avatar
  • 346
0 votes
1 answer
81 views

Can you use Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling?

I have a question regarding the use of Multilevel Modeling (aka Hierarchical Linear Modeling) with Sequential Linear Modeling. I am trying to perform Sequential Linear Modeling (with a binary outcome) ...
Sam16's user avatar
  • 11
4 votes
1 answer
93 views

Is the given procedure SPRT?

In this question, I tried using the fundamental property of the Wald Sequential Probability Ratio Test (SPRT): $P( N = \infty )=0.$ For this I calculated $p_1 = 1- P(-1.5 < S_1 < 2.5 )$, that ...
simran's user avatar
  • 377
0 votes
0 answers
48 views

variance of the sprt stopping time

Is it possible to reduce the variance of the average stopping time of a fixed boundary SPRT by allowing variable boundaries as the test progresses at the price of increasing the average stopping time? ...
hyportnex's user avatar
  • 101
0 votes
0 answers
14 views

Theory behind this natural sequential estimator of the mean

Suppose we have i.i.d. real random variables $X_1, \cdots$. We want to estimate the expectation $E[X_1]$ to within some desired error $\epsilon$, but we do not know the variance. We want to use as few ...
user2316602's user avatar
1 vote
2 answers
856 views

Sequential Bayesian updating of mean and variance of normal distribution

I am trying to write some code to learn the parameters of a normal distribution. I am new to this, and I have patched together the equations from various sources, which may be part of the problem. In ...
Max Montana's user avatar
3 votes
1 answer
87 views

How to parametrize a beta prior for a sequential Bayesian analysis of a proportion?

I'm developing a decision rule for a study using an interim analysis with a Bayesian characterization of the event probability. At a certain point, we will perform a planned stop, and only proceed ...
AdamO's user avatar
  • 63.7k
0 votes
0 answers
88 views

Does magnitude of difference between two variables affect the confidence level?

I have two sets of data that I'm using proprietary software (TubeBuddy) to do data analysis to find which image is "better." I'm assuming the software is doing a two-sample t-test. TubeBuddy'...
Melanie Shebel's user avatar
2 votes
1 answer
167 views

Distribution for Fraction of Success in a Binomial Setting

So the actual original question I am trying to solve is a little bit different than the title: In a binomial setting with probability of success $p$, I keep examining observations until a fraction of ...
Feri's user avatar
  • 197
1 vote
0 answers
29 views

Should SPRT use explicit hypotheses or reflect observed differences in the data?

Sequential Probability Ratio Testing (SPRT) compares the log-likelihood ratio of the observed data under the null and alternative hypotheses to thresholds based on desired alpha and beta levels. I am ...
Joe's user avatar
  • 158
1 vote
0 answers
65 views

Adjusting effect sizes, p values, and CIs for sequential analysis via simulation

It's fairly easy to write a simulation procedure to calculate, for any given global type 1 error rate, adjusted local "stopping" alphas for interim "looks" in a sequential design, ...
gaspar's user avatar
  • 234
2 votes
0 answers
320 views

How is the O'Brien-Fleming constant calculated and how does the procedure control error rates?

O'Brien-Fleming is a commonly used interim analysis procedure in clinical trials. The procedure is used when $N$ tests are performed in sequence. For each test $n \in \{1, ..., N\}$, a test statistic $...
ivan.k's user avatar
  • 41
1 vote
1 answer
28 views

Estimating population means after a selection procedure

The problem is somewhat related to sequential procedures (e.g. Paulson; Gupta and Miescke) I have a hundred engines generating normal random numbers with true (unknown) means $E_1,E_2,E_3...E_{100}$, ...
PKV's user avatar
  • 13
1 vote
0 answers
23 views

When can I stop drawing costly samples?

I'm running (costly) experiments and I need to decide when to stop gathering samples. Is the following approach correct? I'm testing a software package for performance: I measure the time it takes for ...
Xavier Nodet's user avatar
0 votes
0 answers
43 views

I do not understand, are Neyman-Pearson rule and Sequential Probability Ratio Test refers to the same thing?

In Bayesian statistics, I remember to Neyman Pearson rule, in which, based on a threshold value, we have the option of reject to make a decision: decision option. Now, I wonder if it is the same as ...
Mas A's user avatar
  • 223
1 vote
1 answer
276 views

How to prove SPRT terminates with probability unity?

Can anyone provide me the easily explainable proof of 'SPRT terminates with probability unity'? To be more specific- 'Prove that SPRT terminates with probability unity'
Jamil Chowdhury 's user avatar
2 votes
1 answer
69 views

How to explain these jump-discontinuities in the expected stopping time of a SPRT?

Suppose a sequential probability ratio test (SPRT) with $$\alpha=P(\text{Type I error})=P(\text{Type II error})=\beta = 10\%$$ is used to decide between $$\ H_0: p=1/2\quad\text{vs.}\quad H_1: p=b\,$$ ...
r.e.s.'s user avatar
  • 301
2 votes
1 answer
133 views

Changing sample size during experiment? [duplicate]

Will the significance and power levels hold constant if I don't see statistically significant results and decide to wait to allow for more samples to be collected?
ThomasO's user avatar
  • 23
0 votes
2 answers
71 views

When should a problem be approached as a sequential one?

I am working on a problem with a dataset of the form $(x_j,t_i,y_{i,j} )_{(j,i) \in J \times I}$ where $y_{i,j}$ is the output label. Think of progression of illness over time or weather conditions ...
Avatrin's user avatar
  • 92
0 votes
0 answers
95 views

Bayesian sequential updating with current Bayesian sampling software?

I'm having a hard time implementing sequential updating with current software, I don't even know how to start. Basically, I'm having to refit the whole model by simply appending the new data to the ...
user avatar
4 votes
0 answers
81 views

Sequential hypothesis test for uniform distribution

Suppose I have a sequence of data points $x_1,x_2,\dots$. I want to do a sequential hypothesis test, were the null hypothesis is that each $x_i$ is iid uniform on $[0,1]$, and the alternative ...
D.W.'s user avatar
  • 6,688
1 vote
0 answers
59 views

Sequential Testing of Poisson Process

My question is related to the paper "Sequential Testing for Poisson Processes" by Peskir and Shiryaev, available here. Specifically, suppose that there are two states of the world, G(ood) ...
user2660093's user avatar
0 votes
1 answer
360 views

How can I adjust the p-value calculation for an A/B test that is currently running?

I am running an A/B test. I have about 95k samples per variation and 1.29% relative improvement. The test is to see whether the variant converts users to booking better than the control. Test is still ...
bp0308's user avatar
  • 43
0 votes
1 answer
418 views

Group Sequential Clinical Trial Design - Non-binding efficacy boundaries?

I have been reading up on group sequential clinical trial designs and came across the concept of non-binding futility boundaries. Intuitively, it makes sense to me that continuing the trial after ...
random_walk's user avatar
5 votes
2 answers
721 views

Sequential updating of a Bayesian regression model

I'd like to perform Bayesian regression of $Y$ on $X$. i.e. estimate a posterior $p\left(\theta\mid\mathcal{D}\right)$ of some regression parameter $\theta$ given a dataset $\mathcal{D}$ of pairs of ...
H.Rappeport's user avatar
8 votes
1 answer
571 views

How to distinguish between two biased coins

I have two very biased coins: The first coin (C1) lands on heads 95% of the time. The second coin (C2) lands on heads only 1% of the time. One of the coins has been selected and I want to determine ...
WW.'s user avatar
  • 105
3 votes
1 answer
648 views

Term for when hypothesis testing is conducted repeatedly until an interesting result is found?

What is the term for when, through accident or intent, many hypothesis tests are conducted until an 'interesting' result is found? That is, the experiment (or variations on it) are repeated until a ...
stevec's user avatar
  • 303
1 vote
0 answers
408 views

In R, why do the p-values from anova() change when you add more predictors?

When conducting linear regression in R, I am trying to understand how certain p-values are calculated and what they represent. So far this is my understanding: The p-values from summary() correspond ...
M. Smith 's user avatar
6 votes
1 answer
270 views

Equivalence of Contextual Bandit formulations

I find two different types of formulations for Contextual Bandits: Definition 1: In a contextual bandits problem, there is a distribution $P$ over $(x,r_1,...,r_k)$, where $x$ is the context, $a \in [...
Apprentice's user avatar
0 votes
1 answer
291 views

Is sequential probability ratio test bayesian or frequentist or both?

Is sequential probability ratio test bayesian or frequentist or both? Is there a way to implement a bayes decision rule with "sprt"?
user avatar
2 votes
1 answer
55 views

How long will it take to detect that the parameters in a normal distribution have changed, given online sampling?

Let's say, for instance, you have normally distributed data that is coming in one at a time. At some unknown instance, the distribution parameters (mean and variance) change. Assuming that there is ...
user282152's user avatar
2 votes
0 answers
257 views

Sequential testing of hypothesis

Let $\{X_n : n ≥ 1\}$ be a sequence of i.i.d random variables with common density $f_\theta(\cdot )$. Lets we want to test \begin{eqnarray} \begin{array}{cc} H_0: & \theta=\theta_0 \\ H_1: & \...
Masoud's user avatar
  • 1,259
1 vote
0 answers
25 views

Early stopping in wilcoxon signed rank test?

I have decided to acquire N joint samples of variables X and Y (for example, N=200), and test whether their means are different using the signed-rank test. I will repeat this procedure many times, ...
Aleksejs Fomins's user avatar
1 vote
0 answers
1k views

O'Brien Fleming Interim Analysis Interpretation with PROC SEQDESIGN

Quick question about the output of PROC SEQDESIGN. I was playing around with the O'Brien Fleming method trying to understand how we can interpret the results spit out by SAS for a mock study with ...
R. Simian's user avatar
2 votes
1 answer
106 views

How to recursively filter sequential predictions of binary classifier?

Problem My goal is to estimate binary state using entire observation history. Observations are coming sequentially with fixed interval (~1 sec). I have xgboost model which predicts the state given a ...
Stanislav Tsepa's user avatar
2 votes
1 answer
91 views

To improve the posterior belief receiving a time-series from a fixed data source

Let data $\mathbf{X}\in \mathbb{R}^d$ come from one of the $K$ possible sources $\mathsf{S}\in \{1,2,...,K\}$. The true $\mathsf{S}$ is unknown but it is fixed. The main task is to infer the true ...
Mo-'s user avatar
  • 526
1 vote
1 answer
38 views

extension of sequential probability ratio tests to particle filters?

I've been wondering if there are extensions of the sequential probability ratio test to account for particle filters. I ask because, in my research, I'm working with distributions that cannot be ...
Kashif's user avatar
  • 517