# Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

3,570 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1k views

### Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
1k views

### Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
204 views

### Unit testing sampling methods

I'm writing a bit of code that makes pretty heavy use of sampling (eg, MCMC, Particle Filters, etc), and I would really like to test it to make sure that it's doing what I think it is before claiming ...
434 views

### How is it possible to control false discovery rate (FDR) without knowing the power and the prevalence of the nulls?

If we have p-value 0.05, to calculate probability of our discovery to be false positive, we need to use complex formula with prevalence (prior probability) and statistical power. If we have lots of ...
2k views

### How to test for the difference in skewness of two samples?

I have two samples. From looking at their densities, one appears symmetrical and the other from some right-tailed distribution. I would like to test that the two do not have the same skewness (...
1k views

### How to handle multiple measurements per participant, with categorical data?

I've done an experiment where I've collected measurements from a number of participants. Each relevant data point has two variables, both categorical: in fact, each variable has two possible values (...
661 views

### Are these equivalent (for p-values): super-uniform, stochastically larger than / dominating the uniform, conservative?

In the literature and online, I have found three different wordings that I think refer to the same concept: stochastically larger than uniform (which I take is ...
3k views

5k views

### How to analyze and interpret A/B test results via Bootstrap method?

We've run a split test of a new product feature and want to measure if the uplift on revenue is significant. Our observations are definitely not normally distributed (most of our users don't spend, ...
4k views

### Comparing two F-test statistics

I have two groups A and B, each of which consists of 5 samples. Each sample is described in a vector of length (>1000) of continuous numeric values (characteristics). I want to test if the sample in ...
248 views

### The Mediterranean diet - statistics issues with the updated paper

In 2013, a group of researchers published a paper with results on a randomized trial of the Mediterranean diet, finding that it appears to have significant health benefits. Today, they retracted ...
1k views

### Combining Z-scores using Stouffer's approach

Quick background: I require to obtain a single 'score' for comparison between experimental and simulated data. The parameter being tested is the height of various parts (num = 5) of the model. For the ...
2k views

### 2-Sample Proportions z-test vs Fisher's Exact Test

I want to evaluate the difference in proportions in declined applications and exceptions from policy from one quarter of data to the next. My sample sizes are generally not small (several hundred to ...
380 views

### Is this popular approach to Bootstrap hypothesis testing correct?

A widely cited paper "Statistical Significance Tests for Machine Translation Evaluation" proposes usage of Bootstrap for evaluating significance of difference across systems for Machine Translation (...
628 views

### Skew of p-value distribution under composite null hypotheses

On this page it says ...if HA holds, the p-values have a distribution for which values near 0 are more likely than values near 1. However the p-values may have a distribution that is not ...
162 views

164 views

### Exact Null Distribution with Ties

I am interested in deriving exact null distributions for small-sample test statistics with non-trivial ties. Not fundamentally continuous variables that happen to have a few repeated values, but ...
114 views

### Statistical test to see whether we drew overly-balanced samples

I have an urn with $R$ red and $B$ black balls. $R$ and $B$ are large numbers. I draw some $n$ samples of different sizes without replacement until the urn is empty. The sizes of the samples are not a ...
664 views

### Hypothesis testing for Gamma distribution

I have a sample $X_1,...,X_n \sim \Gamma(\alpha, \beta)$, where $\alpha, \beta$ - unknown parameters of Gamma distribution. How to build a test for testing $H_0:\alpha=1$ against $H_1:\alpha>1$?
378 views

703 views

### appropriate hypothesis test for non-negative quantity

This question is intentionally written in a very abstract way, because I am interested in a generic solution, or for pointers to different possible concrete cases in different fields: I am measuring ...
410 views

### How Should I Find Statistically Significant Differences Between Two Markov Models?

Suppose I have N Markov Models of M states representing the behaviour patterns of 2 different groups (note: fully observable models, no hidden states), and have stored each model as a matrix of ...
2k views

751 views

### Confirming the detailed calcs for NPS Margin of error

Based on this very good post How can I calculate margin of error in a NPS (Net Promoter Score) result? I've pulled together the detailed calculations to perform the test for the generalised NPS ...
1k views

### Binomial mid-p value

I've been under the impression that the mid-$p$ values generally control the Type I error, and consequently confidence intervals based on mid-$p$ values control the coverage. However I have checked ...
788 views

### Using the hypergeometric distribution for skipping events in transcriptome sequencing

My question is inspired by this post. However, its a bit more complicated than that to explain. I hope I succeed. I work with RNA-Seq data on alternative splicing in plants. For this discussion, lets ...
71 views

### How does R's aov function 'work better' with orthogonal contrasts?

The help page for R's aov function states: The default ‘contrasts’ in R are not orthogonal contrasts, and aov and its helper functions will work better with such ...
I've been studying more about GLRT (Generalized Likelihood Ratio Tests) and I came up with the following problem. Let $X\sim N(\theta,1)$ and consider the hypothesis $H_0:\theta\in[a,b]$ against \$H_1:\...