Questions tagged [sequential-analysis]

Hypothesis testing without a fixed sample size.

Filter by
Sorted by
Tagged with
1 vote
0 answers
23 views

Distribution for Fraction of Success in a Binomial Setting

So the actual original question I am trying to solve is a little bit different than the title: In a binomial setting with probability of success $p$, I keep examining observations until a fraction of ...
user avatar
  • 161
0 votes
0 answers
21 views

Expected sample size for sequential t-test

I have a 2 group sequential one-sided t-test setting. $1-\beta = 90\%$, $\alpha = 5\%$. Null hypothesis is $\mu_1=\mu_2=1800$. Alternative hypothesis is: $\mu_1\geq\mu_2$ Plug-in estimate of $\sigma$ ...
user avatar
  • 1,220
2 votes
1 answer
39 views

Detection of 'unfair' Bernoulli sequences using run lengths?

I'm rather confused about the following question: Which of the following vectors do you think is actually a random sample of $\mathcal{B}(1, 0.5)$ (Bernoulli) of length 10? $[0,1,1,0,1,0,1,0,1,1]$ and ...
user avatar
1 vote
0 answers
12 views

Should SPRT use explicit hypotheses or reflect observed differences in the data?

Sequential Probability Ratio Testing (SPRT) compares the log-likelihood ratio of the observed data under the null and alternative hypotheses to thresholds based on desired alpha and beta levels. I am ...
user avatar
  • 146
0 votes
0 answers
22 views

Adjusting effect sizes, p values, and CIs for sequential analysis via simulation

It's fairly easy to write a simulation procedure to calculate, for any given global type 1 error rate, adjusted local "stopping" alphas for interim "looks" in a sequential design, ...
user avatar
  • 224
1 vote
0 answers
18 views

How is the O'Brien-Fleming constant calculated and how does the procedure control error rates?

O'Brien-Fleming is a commonly used interim analysis procedure in clinical trials. The procedure is used when $N$ tests are performed in sequence. For each test $n \in \{1, ..., N\}$, a test statistic $...
user avatar
  • 31
1 vote
1 answer
25 views

Estimating population means after a selection procedure

The problem is somewhat related to sequential procedures (e.g. Paulson; Gupta and Miescke) I have a hundred engines generating normal random numbers with true (unknown) means $E_1,E_2,E_3...E_{100}$, ...
user avatar
  • 13
0 votes
0 answers
31 views

I do not understand, are Neyman-Pearson rule and Sequential Probability Ratio Test refers to the same thing?

In Bayesian statistics, I remember to Neyman Pearson rule, in which, based on a threshold value, we have the option of reject to make a decision: decision option. Now, I wonder if it is the same as ...
user avatar
  • 189
0 votes
0 answers
12 views

Sequential Probability Ratio Test (Property Proof)

Can anyone provide me the easily explainable proof of 'SPRT terminates with probability unity'? To be more specific- 'Prove that SPRT terminates with probability unity'
user avatar
1 vote
1 answer
49 views

How to explain these jump-discontinuities in the expected stopping time of a SPRT?

Suppose a sequential probability ratio test (SPRT) with $$\alpha=P(\text{Type I error})=P(\text{Type II error})=\beta = 10\%$$ is used to decide between $$\ H_0: p=1/2\quad\text{vs.}\quad H_1: p=b\,$$ ...
user avatar
  • 281
2 votes
1 answer
34 views

Changing sample size during experiment? [duplicate]

Will the significance and power levels hold constant if I don't see statistically significant results and decide to wait to allow for more samples to be collected?
user avatar
  • 23
0 votes
2 answers
59 views

When should a problem be approached as a sequential one?

I am working on a problem with a dataset of the form $(x_j,t_i,y_{i,j} )_{(j,i) \in J \times I}$ where $y_{i,j}$ is the output label. Think of progression of illness over time or weather conditions ...
user avatar
  • 112
0 votes
0 answers
51 views

Bayesian sequential updating with current Bayesian sampling software?

I'm having a hard time implementing sequential updating with current software, I don't even know how to start. Basically, I'm having to refit the whole model by simply appending the new data to the ...
user avatar
3 votes
0 answers
38 views

Sequential hypothesis test for uniform distribution

Suppose I have a sequence of data points $x_1,x_2,\dots$. I want to do a sequential hypothesis test, were the null hypothesis is that each $x_i$ is iid uniform on $[0,1]$, and the alternative ...
user avatar
  • 5,953
1 vote
0 answers
37 views

Sequential Testing of Poisson Process

My question is related to the paper "Sequential Testing for Poisson Processes" by Peskir and Shiryaev, available here. Specifically, suppose that there are two states of the world, G(ood) ...
user avatar
0 votes
1 answer
80 views

How can I adjust the p-value calculation for an A/B test that is currently running?

I am running an A/B test. I have about 95k samples per variation and 1.29% relative improvement. The test is to see whether the variant converts users to booking better than the control. Test is still ...
user avatar
  • 33
0 votes
1 answer
110 views

Group Sequential Clinical Trial Design - Non-binding efficacy boundaries?

I have been reading up on group sequential clinical trial designs and came across the concept of non-binding futility boundaries. Intuitively, it makes sense to me that continuing the trial after ...
user avatar
2 votes
1 answer
136 views

Sequential updating of a Bayesian regression model

I'd like to perform Bayesian regression of $Y$ on $X$. i.e. estimate a posterior $p\left(\theta\mid\mathcal{D}\right)$ of some regression parameter $\theta$ given a dataset $\mathcal{D}$ of pairs of ...
user avatar
8 votes
1 answer
340 views

How to distinguish between two biased coins

I have two very biased coins: The first coin (C1) lands on heads 95% of the time. The second coin (C2) lands on heads only 1% of the time. One of the coins has been selected and I want to determine ...
user avatar
  • 105
3 votes
1 answer
156 views

Term for when hypothesis testing is conducted repeatedly until an interesting result is found?

What is the term for when, through accident or intent, many hypothesis tests are conducted until an 'interesting' result is found? That is, the experiment (or variations on it) are repeated until a ...
user avatar
  • 259
1 vote
0 answers
47 views

In R, why do the p-values from anova() change when you add more predictors?

When conducting linear regression in R, I am trying to understand how certain p-values are calculated and what they represent. So far this is my understanding: The p-values from summary() correspond ...
user avatar
5 votes
1 answer
138 views

Equivalence of Contextual Bandit formulations

I find two different type of Contextual Bandit problem formulations in the literature: Definition 1: (https://hunch.net/~jl/projects/interactive/sidebandits/bandit.pdf) In a contextual bandits problem,...
user avatar
0 votes
1 answer
157 views

Is sequential probability ratio test bayesian or frequentist or both?

Is sequential probability ratio test bayesian or frequentist or both? Is there a way to implement a bayes decision rule with "sprt"?
user avatar
2 votes
1 answer
51 views

How long will it take to detect that the parameters in a normal distribution have changed, given online sampling?

Let's say, for instance, you have normally distributed data that is coming in one at a time. At some unknown instance, the distribution parameters (mean and variance) change. Assuming that there is ...
user avatar
1 vote
0 answers
19 views

Early stopping in wilcoxon signed rank test?

I have decided to acquire N joint samples of variables X and Y (for example, N=200), and test whether their means are different using the signed-rank test. I will repeat this procedure many times, ...
user avatar
1 vote
0 answers
557 views

O'Brien Fleming Interim Analysis Interpretation with PROC SEQDESIGN

Quick question about the output of PROC SEQDESIGN. I was playing around with the O'Brien Fleming method trying to understand how we can interpret the results spit out by SAS for a mock study with ...
user avatar
2 votes
1 answer
68 views

How to recursively filter sequential predictions of binary classifier?

Problem My goal is to estimate binary state using entire observation history. Observations are coming sequentially with fixed interval (~1 sec). I have xgboost model which predicts the state given a ...
user avatar
2 votes
1 answer
91 views

To improve the posterior belief receiving a time-series from a fixed data source

Let data $\mathbf{X}\in \mathbb{R}^d$ come from one of the $K$ possible sources $\mathsf{S}\in \{1,2,...,K\}$. The true $\mathsf{S}$ is unknown but it is fixed. The main task is to infer the true ...
user avatar
  • 438
1 vote
1 answer
33 views

extension of sequential probability ratio tests to particle filters?

I've been wondering if there are extensions of the sequential probability ratio test to account for particle filters. I ask because, in my research, I'm working with distributions that cannot be ...
user avatar
0 votes
1 answer
99 views

Binary Sequence Prediction Model with Time dependant features

I got a very long sequence of binary items (0 or 1). Each item is associated to a timestamp. For example : ...
user avatar
5 votes
1 answer
667 views

What are disadvantages of "Sequential analysis"

Sequential analysis involves performing sequential interim analysis till results are significant or till a maximum number of interim analyses is reached. Sequential analysis sounds appealing ...
user avatar
  • 8,993
1 vote
0 answers
31 views

Average Sample Number using Page's CUSUM procedure?

I am trying reproduce the Table 1 results from the page 12 using the equation given the page 13.To access the journal article please click https://arxiv.org/pdf/math/0605322.pdf. The corresponding ...
user avatar
  • 335
3 votes
1 answer
165 views

Does an interim sample size re-estimation increase type 1 error if based on the overall event rate?

Suppose that a sample size is to be calculated for a trial: a presumed effect size summarizes the effect of intervention (relative risk RR $\exp(\theta_1)$), and background gives the rate of events in ...
user avatar
  • 53k
2 votes
0 answers
55 views

Mixed-effects modelling in a sequential analysis design

I'm planning a study and am considering going with a sequential analysis design (as described here: https://psyarxiv.com/9yegd/download?format=pdf). This design allows one to take multiple looks at ...
user avatar
  • 63
0 votes
0 answers
26 views

How to methodologically show that a given 'time-series/sequential' data is not really sequential?

I have an apparent time-series/sequencial (supervised: multi-class classification problem) dataset with each data-point time-stamped. However, some domain intuition tells me that the data is static ...
user avatar
  • 89
-1 votes
1 answer
536 views

Reverse the shuffling phase

It is very convenient to have sklearn library to split and shuffle the data with ...
user avatar
4 votes
1 answer
117 views

Can the Bayesian but not the frequentist "just add more observations"?

Since the frequentist's p-values are uniformly distributed under the null hypothesis, it is a highly problematic practice to add more and more data to your sample until you find a significant result. ...
user avatar
  • 79
1 vote
0 answers
14 views

Sequential mining technique for categorical events?

I'm facing a problem where I have a short (2-8) sequence of discrete events, of variable length, and a final state. Ex.: cold, warm, warm -> soft clothing. I would like to determine what are the main ...
user avatar
  • 11
2 votes
0 answers
77 views

Predicting elections with a random sample

If I have a population of exactly 100.000 voters, and I know they all voted in either candidate A, B, C, D or E. (e.g where the results of a random sample of 100 voters are A = 40%, B = 30%, C = 10%, ...
user avatar
1 vote
1 answer
49 views

Applying Sequential Probability Ratio Test (SPRT) on a Vector Signal?

I am trying to apply SPRT on a vector signal (of residual) for detecting faults occurring in the sensors' measurement of an aircraft. The vector signal contains measurements from five different ...
user avatar
  • 11
6 votes
1 answer
278 views

Stopping rules in group sequential designs

I have read that in group sequential designs, having a maximum of $k$ stops, for the $j$-th stop $j < k$ there are 4 bounds chosen $a_j,b_j, c_j, d_j$ such that the three decisions are considered. ...
user avatar
  • 53k
2 votes
0 answers
30 views

power calculation for two stage binomial type model

This is a power-type calculation for a Bernoulli/binomial question in two stages. Suppose you are planning an experiment which starts with a test for an event on $N$ experimental units. The event ...
user avatar
  • 343
12 votes
5 answers
2k views

Determining sample size before starting an experiment or run the experiment indefinitely?

I studied statistics years ago and have forgotten it all so these may seem like general conceptual questions than anything specific but here is my issue. I work for an e-commerce website as a UX ...
user avatar
  • 221
4 votes
1 answer
2k views

How many samples do I need to estimate a gaussian distribution?

Background: I'm comparing the performance of two algorithms. The data is noisy so the results tend to form a nice normal distribution. I'd like to estimate the mean with a high confidence: up to 1% ...
user avatar
  • 41
3 votes
0 answers
28 views

Advice for conducting the same experiment several times?

I'm very new to Bayesian statistics, but I've found myself in a situation where they might be able to help. I'll be running the same experiment repeatedly and analyzing the data every few months. The ...
user avatar
  • 273
2 votes
0 answers
422 views

Understanding interim analysis and alpha spending functions

I have to perform an A/B test with interim analysis and found some literature about methods designed to preserve the global Type I error rate by adjusting the $\alpha$ level for each peeking. ...
user avatar
  • 143
1 vote
0 answers
76 views

Convergence of a sequence in finite number of steps

Here is the setup of my problem. It is a sequential problem and there are two possible actions A and B. Now, when either action $A$ or $B$ is taken at the $j$th time point, we observe some outcome say ...
user avatar
2 votes
0 answers
49 views

Optimal denoising of a few features for latent variable modeling

I am fitting latent variable models to noisy, high-dimensional data. The dimension is 15,000 to 20,000; there are 2,000 observations. There is a certain latent structure that I observe -- essentially, ...
user avatar
1 vote
1 answer
917 views

Bayesian updating with discrete priors + possibly unknown classes

I'm following along with some lecture notes on Bayesian updating with discrete priors. They give an example problem to illustrate some of these concepts, which I briefly restate here: Someone tells ...
user avatar
1 vote
1 answer
391 views

sample size calculation for one-sample log rank test with interims (ie group sequential)

I wonder if there is any advice for performing a sample size calculation for survival data using the one-sample log rank test with interims (ie group sequential) - ie with O'Brien-Fleming or some ...
user avatar