# Questions tagged [sequential-analysis]

Hypothesis testing without a fixed sample size.

47 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1k views

### Bayesian Q-learning

Suppose that, for every state $s$, there is a set of actions $\mathcal{A}(s)$ that can be chosen in that state. Let $Q(s, a)$ denote the expected utility of choosing action $a \in \mathcal{A}(s)$ in ...
915 views

### Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
180 views

### bias of an estimator when using stopping rules

Consider the setting where $X_1,X_2,...$ are i.i.d. real-valued random variables with $\mathbb{E}[X_i] = \theta$ and let the random variable $\tau$ be an associated stopping time. I'm wondering what ...
20 views

### Sequential hypothesis test for uniform distribution

Suppose I have a sequence of data points $x_1,x_2,\dots$. I want to do a sequential hypothesis test, were the null hypothesis is that each $x_i$ is iid uniform on $[0,1]$, and the alternative ...
26 views

### Advice for conducting the same experiment several times?

I'm very new to Bayesian statistics, but I've found myself in a situation where they might be able to help. I'll be running the same experiment repeatedly and analyzing the data every few months. The ...
108 views

### How to reconcile results from many incremental hypothesis tests?

I'm considering a series of controlled experiments (e.g. A/B tests) measuring the performance of a system. Each time a change to the system is found significant (via statistical inference) the test ...
135 views

### Estimation on evolving distribution with small updates

I have a set $X$ of $10^6$ elements and a time series of probability distributions $\mu_1,\mu_2,\ldots$ on $X$. I want to estimate the expected value of a function $f$ over each $\mu_t$. It is easy to ...
29 views

### Hypothesis test based on (random) result of previous experiment: can I multiply confidence levels?

Consider an i.i.d. sequence of Bernoulli random variables $\{Z_i\}$ with Bernoulli parameter $x$. By a certain procedure (experiment A) I obtain a random set S such that $\mathrm{Pr}[x \in S] \geq c_A$...
1k views

### Sequential Probability Ratio Test (or other Sequential Sampling techniques) for testing difference

I have the results from running two algorithms and I want to be able to say that there is, say, a 95% probability that one of the sets of results is different to the other where different means A > B ...
76 views

### Running sequential tests as data is collected - practical advice for researchers

I've observed that many experimental labs in my field don't do a power analysis before they start collecting data. In practice, it seems that many researchers collect data and periodically run ...
48 views

### Mixed-effects modelling in a sequential analysis design

I'm planning a study and am considering going with a sequential analysis design (as described here: https://psyarxiv.com/9yegd/download?format=pdf). This design allows one to take multiple looks at ...
63 views

### Predicting elections with a random sample

If I have a population of exactly 100.000 voters, and I know they all voted in either candidate A, B, C, D or E. (e.g where the results of a random sample of 100 voters are A = 40%, B = 30%, C = 10%, ...
28 views

### power calculation for two stage binomial type model

This is a power-type calculation for a Bernoulli/binomial question in two stages. Suppose you are planning an experiment which starts with a test for an event on $N$ experimental units. The event ...
348 views

### Understanding interim analysis and alpha spending functions

I have to perform an A/B test with interim analysis and found some literature about methods designed to preserve the global Type I error rate by adjusting the $\alpha$ level for each peeking. ...
47 views

### Optimal denoising of a few features for latent variable modeling

I am fitting latent variable models to noisy, high-dimensional data. The dimension is 15,000 to 20,000; there are 2,000 observations. There is a certain latent structure that I observe -- essentially, ...
27 views

### Just how bad is it to determine when to stop data collection based on the resulting CI?

Say I want to be $95\%$ sure of the mean $\bar X$, with a CI of size $2$ ($1$ in each direction). The following is bad practice: collect data until $1.96\frac{s}{\sqrt{n}}=1$. What is the extent ...
49 views

### What type of analysis for sequential data can I use to identify patterns in collaborative activities

I have a data set, which is collected every 5 second for a total of one hour using a special type of wearable device. Each data point is the amount of verbal participation ratio (measured as percent) ...
84 views

### Random walk with continuously distributed steps on $[-1,1]$

A simple random walk $S_n = X_1 +\cdots +X_n$, where $P(X_i = 1) = p \not = 0.5$ and $P(X_i=-1)= q \triangleq 1-p$, admits the following probability P(S_n \textrm{ reaches } a \textrm{ before} -b) =...
202 views

### How can Neyman-Pearson and Wald both be optimal?

I see optimality mentioned in the context of both Neyman-Pearson tests and Wald tests. Neyman Pearson is a fixed sample test, while Wald is a sequential test. The Neyman-Pearson lemma states that no ...
42 views

### Interim analysis and CI

A trial was planned and designed to be a single arm trial (one group), and the sample size was calculating using a confidence interval to achieve some accuracy in the estimation of the unknown ...
237 views