Questions tagged [sequential-analysis]

Hypothesis testing without a fixed sample size.

47 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
14
votes
0answers
1k views

Bayesian Q-learning

Suppose that, for every state $s$, there is a set of actions $\mathcal{A}(s)$ that can be chosen in that state. Let $Q(s, a)$ denote the expected utility of choosing action $a \in \mathcal{A}(s)$ in ...
10
votes
0answers
915 views

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
6
votes
0answers
180 views

bias of an estimator when using stopping rules

Consider the setting where $X_1,X_2,...$ are i.i.d. real-valued random variables with $\mathbb{E}[X_i] = \theta$ and let the random variable $\tau$ be an associated stopping time. I'm wondering what ...
3
votes
0answers
20 views

Sequential hypothesis test for uniform distribution

Suppose I have a sequence of data points $x_1,x_2,\dots$. I want to do a sequential hypothesis test, were the null hypothesis is that each $x_i$ is iid uniform on $[0,1]$, and the alternative ...
3
votes
0answers
26 views

Advice for conducting the same experiment several times?

I'm very new to Bayesian statistics, but I've found myself in a situation where they might be able to help. I'll be running the same experiment repeatedly and analyzing the data every few months. The ...
3
votes
1answer
108 views

How to reconcile results from many incremental hypothesis tests?

I'm considering a series of controlled experiments (e.g. A/B tests) measuring the performance of a system. Each time a change to the system is found significant (via statistical inference) the test ...
3
votes
0answers
135 views

Estimation on evolving distribution with small updates

I have a set $X$ of $10^6$ elements and a time series of probability distributions $\mu_1,\mu_2,\ldots$ on $X$. I want to estimate the expected value of a function $f$ over each $\mu_t$. It is easy to ...
3
votes
0answers
29 views

Hypothesis test based on (random) result of previous experiment: can I multiply confidence levels?

Consider an i.i.d. sequence of Bernoulli random variables $\{Z_i\}$ with Bernoulli parameter $x$. By a certain procedure (experiment A) I obtain a random set S such that $\mathrm{Pr}[x \in S] \geq c_A$...
3
votes
1answer
1k views

Sequential Probability Ratio Test (or other Sequential Sampling techniques) for testing difference

I have the results from running two algorithms and I want to be able to say that there is, say, a 95% probability that one of the sets of results is different to the other where different means A > B ...
3
votes
0answers
76 views

Running sequential tests as data is collected - practical advice for researchers

I've observed that many experimental labs in my field don't do a power analysis before they start collecting data. In practice, it seems that many researchers collect data and periodically run ...
2
votes
0answers
48 views

Mixed-effects modelling in a sequential analysis design

I'm planning a study and am considering going with a sequential analysis design (as described here: https://psyarxiv.com/9yegd/download?format=pdf). This design allows one to take multiple looks at ...
2
votes
0answers
63 views

Predicting elections with a random sample

If I have a population of exactly 100.000 voters, and I know they all voted in either candidate A, B, C, D or E. (e.g where the results of a random sample of 100 voters are A = 40%, B = 30%, C = 10%, ...
2
votes
0answers
28 views

power calculation for two stage binomial type model

This is a power-type calculation for a Bernoulli/binomial question in two stages. Suppose you are planning an experiment which starts with a test for an event on $N$ experimental units. The event ...
2
votes
0answers
348 views

Understanding interim analysis and alpha spending functions

I have to perform an A/B test with interim analysis and found some literature about methods designed to preserve the global Type I error rate by adjusting the $\alpha$ level for each peeking. ...
2
votes
0answers
47 views

Optimal denoising of a few features for latent variable modeling

I am fitting latent variable models to noisy, high-dimensional data. The dimension is 15,000 to 20,000; there are 2,000 observations. There is a certain latent structure that I observe -- essentially, ...
2
votes
0answers
27 views

Just how bad is it to determine when to stop data collection based on the resulting CI?

Say I want to be $95\%$ sure of the mean $\bar X$, with a CI of size $2$ ($1$ in each direction). The following is bad practice: collect data until $1.96\frac{s}{\sqrt{n}}=1$. What is the extent ...
2
votes
0answers
49 views

What type of analysis for sequential data can I use to identify patterns in collaborative activities

I have a data set, which is collected every 5 second for a total of one hour using a special type of wearable device. Each data point is the amount of verbal participation ratio (measured as percent) ...
2
votes
0answers
84 views

Random walk with continuously distributed steps on $[-1,1]$

A simple random walk $S_n = X_1 +\cdots +X_n$, where $P(X_i = 1) = p \not = 0.5$ and $P(X_i=-1)= q \triangleq 1-p$, admits the following probability $$P(S_n \textrm{ reaches } a \textrm{ before} -b) =...
2
votes
0answers
202 views

How can Neyman-Pearson and Wald both be optimal?

I see optimality mentioned in the context of both Neyman-Pearson tests and Wald tests. Neyman Pearson is a fixed sample test, while Wald is a sequential test. The Neyman-Pearson lemma states that no ...
2
votes
0answers
42 views

Interim analysis and CI

A trial was planned and designed to be a single arm trial (one group), and the sample size was calculating using a confidence interval to achieve some accuracy in the estimation of the unknown ...
2
votes
0answers
237 views

rejection region for correlated bivariate normal

STATEMENT OF PROBLEM: Suppose $ \left( \begin{array}{ccc} \ Z_1 \\ Z_2 \end{array} \right)$ follows a Bivariate standard normal with covariance $ \rho $ $ \left( \begin{array}{ccc} \ Z_1 \\ Z_2 \...
1
vote
0answers
23 views

Sequential Testing of Poisson Process

My question is related to the paper "Sequential Testing for Poisson Processes" by Peskir and Shiryaev, available here. Specifically, suppose that there are two states of the world, G(ood) ...
1
vote
1answer
57 views

Sequential updating of a Bayesian regression model

I'd like to perform Bayesian regression of $Y$ on $X$. i.e. estimate a posterior $p\left(\theta\mid\mathcal{D}\right)$ of some regression parameter $\theta$ given a dataset $\mathcal{D}$ of pairs of ...
1
vote
0answers
29 views

In R, why do the p-values from anova() change when you add more predictors?

When conducting linear regression in R, I am trying to understand how certain p-values are calculated and what they represent. So far this is my understanding: The p-values from summary() correspond ...
1
vote
0answers
19 views

Early stopping in wilcoxon signed rank test?

I have decided to acquire N joint samples of variables X and Y (for example, N=200), and test whether their means are different using the signed-rank test. I will repeat this procedure many times, ...
1
vote
0answers
326 views

O'Brien Fleming Interim Analysis Interpretation with PROC SEQDESIGN

Quick question about the output of PROC SEQDESIGN. I was playing around with the O'Brien Fleming method trying to understand how we can interpret the results spit out by SAS for a mock study with ...
1
vote
0answers
26 views

Average Sample Number using Page's CUSUM procedure?

I am trying reproduce the Table 1 results from the page 12 using the equation given the page 13.To access the journal article please click https://arxiv.org/pdf/math/0605322.pdf. The corresponding ...
1
vote
0answers
14 views

Sequential mining technique for categorical events?

I'm facing a problem where I have a short (2-8) sequence of discrete events, of variable length, and a final state. Ex.: cold, warm, warm -> soft clothing. I would like to determine what are the main ...
1
vote
1answer
37 views

Applying Sequential Probability Ratio Test (SPRT) on a Vector Signal?

I am trying to apply SPRT on a vector signal (of residual) for detecting faults occurring in the sensors' measurement of an aircraft. The vector signal contains measurements from five different ...
1
vote
0answers
61 views

Convergence of a sequence in finite number of steps

Here is the setup of my problem. It is a sequential problem and there are two possible actions A and B. Now, when either action $A$ or $B$ is taken at the $j$th time point, we observe some outcome say ...
1
vote
0answers
30 views

maximum cumulative decision limit

It is well-known that it is problematic to stop an A/B test before we obtain a required number of samples. This website "A/B Testing Rigorously (without losing your job)" explains how to stop safely ...
1
vote
0answers
64 views

Terminology for sequential estimation with randomized stopping time

Consider a sequence of i.i.d Bernoulli RV's $\{X_i\}_{i \in \mathbb N}$ with parameter $p$. Based on this sequence we build a sequential estimator of $p$ (using inverse binomial sampling for example). ...
1
vote
0answers
193 views

What are few techniques that use sequential pattern mining for predicting location?

I found many sequential pattern techniques like GSP Algorithm Sequential PАttern Discovery using Equivalence classes (SPADE) Apriori algorithm etc But they all work for data with certainty like ...
1
vote
0answers
54 views

How can I maintain a streaming value and be 95% confident that the measure for the population dropped below some value?

I weigh hail stones one at a time, producing a sequence of weights. I want to tell the weatherman when I'm 95% confident that hail stones are less than 50g in weight based on the previous $K$ hail ...
1
vote
0answers
269 views

One pass distributed clustering algorithm

I am working on clustering of data streams. For my purpose Sequential Leader Clustering (SLC) have given fair result, as i need not give number of clusters (like k in k means), which require some ...
0
votes
0answers
30 views

Bayesian sequential updating with current Bayesian sampling software?

I'm having a hard time implementing sequential updating with current software, I don't even know how to start. Basically, I'm having to refit the whole model by simply appending the new data to the ...
0
votes
0answers
8 views

good quotation on the control part of statistical quality control?

I'm writing a paper making use of techniques from statistical quality control, and would like to include a quotation that expresses a sentiment something like this: Statistical quality control is not ...
0
votes
0answers
17 views

Group Sequential Clinical Trial Design - Non-binding efficacy boundaries?

I have been reading up on group sequential clinical trial designs and came across the concept of non-binding futility boundaries. Intuitively, it makes sense to me that continuing the trial after ...
0
votes
0answers
26 views

How to methodologically show that a given 'time-series/sequential' data is not really sequential?

I have an apparent time-series/sequencial (supervised: multi-class classification problem) dataset with each data-point time-stamped. However, some domain intuition tells me that the data is static ...
0
votes
1answer
37 views

Finding expected value with $3$ unknown variables

Question: There are n marbles in a bag, player $1$ picks out $x$ marbles where $x \leq n$, marks them, and puts them back. Player $2$ picks out $y$ marbles where $y \leq n$. What is the expected ...
0
votes
0answers
17 views

Derive sequencing probabilities from univariate data

Given a variable measured sequentially e.g. -6,-3,0,2,-3,1,-5,7,-47.. etc, what are some methods to derive some reliable sequencing probabilities? e.g. if i-2>0 and i-1 >0 then i>0 etc.
0
votes
0answers
24 views

Is there a statistical correction that works if the number of experiments depends on the result of the experiments?

Suppose that I am running an experiment that gives us some confidence $c$ about a hypothesis. If I intend to run the experiment $n$ times, I understand there are some ways of choosing a $c'$ so that ...
0
votes
0answers
89 views

changepoint detection and its analysis

I applied changepoint detection provided by ecp package against a given time series. The time series plot marked with identified change points is shown as follows. ...
0
votes
1answer
18 views

Optimally sampling a sequence of observations

Given a multivariate population, I'd like to define a sequence of observations to sample, such that coverage and representativeness is optimal at each step (cumulatively). For example, a sample of ...
0
votes
0answers
46 views

Closed form Pocock Rejection Region solution for joint MVN correlated RVs?

This is related to my previous question (link), but I have a simpler way of expressing it. Suppose $ \left( \begin{array}{ccc} \ Z_1 \\ Z_2 \end{array} \right)$ follows a Bivariate standard normal ...
0
votes
0answers
35 views

Statistical similarity on sequences

What are the methods that can be used for evaluating the similarity of the sequences? Lets say that we have three sequences; I want to be able to say that "statistically" the sequence number 1 and the ...
0
votes
0answers
217 views

Sequential Proportions Test

I have been trying to find some R code but haven't had any luck so far. I am able to perform a sequential t-test on some data that I was working on previously but now my data has a combination of real ...