Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

2,322 questions
2k views

When does a UMP test fail to exist?

I have a sample $X=(X_1, ...,X_n)\sim N(\mu,\sigma^2)$ with $\sigma^2$ known. The hypotheses are $H_0: \mu=\mu_0, H_1:\mu \neq \mu_0$. I know that in such a case an UMP test does not exist and so ...
958 views

Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
830 views

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
130 views

Is the Berry-Esseen theorem useful for justifying normality?

The Kolmogorov-Smirnov (KS) test tells one how confident they can be that a sample comes from a hypothesized distribution. It is my understanding that this test can be used to justify whether or not ...
177 views

Testing for Benford Law in real time

Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution ...
559 views

Skew of p-value distribution under composite null hypotheses

On this page it says ...if HA holds, the p-values have a distribution for which values near 0 are more likely than values near 1. However the p-values may have a distribution that is not ...
46 views

Statistical test to see whether we drew overly-balanced samples

I have an urn with $R$ red and $B$ black balls. $R$ and $B$ are large numbers. I draw some $n$ samples of different sizes without replacement until the urn is empty. The sizes of the samples are not a ...
790 views

Use Chi-Squared or Binomial Test if Distribution is not Known?

Suppose you have a set of data (eg. [a, b, a, a, b, b, etc.]), and you have the suspicion that the set of data follows the binomial distribution. Your Null Hypothesis is: The probability of success ...
404 views

Hypothesis testing for binomial distribution: a simple real-life case

I'm refreshing (or maybe just acquiring) some stats skills, and I have a real-life situation, probably very simple since it's quite close to a typical example from stats courses, that I would like to ...
2k views

532 views

Hypothesis testing with non-iid observations

I have a dataset with observations which violate the independent and identically distributed (iid) assumption. The dataset is composed by a number of post-click conversion events (purchases on a ...
193 views

147 views

53 views

How is it possible to control false discovery rate (FDR) without knowing the power and the prevalence of the nulls?

If we have p-value 0.05, to calculate probability of our discovery to be false positive, we need to use complex formula with prevalence (prior probability) and statistical power. If we have lots of ...
124 views

Comparing multiple frequency distributions from data and from simulation

I am attempting to analyze the shuffling in the card based online game Magic Arena, which I suspect has a specific bug that causes the distribution of outputs to be biased. Is the approach I have ...
78 views

Can I statistically describe a single case/outlier vs. a distribution?

I have a dataset consisting of body weight and corresponding age for a bunch of healthy subjects (grey triangles below). I fit a nonlinear function to this data and graphed a 95% prediction interval. ...
65 views

A possible typo in the textbook?

On page 74 of Lehmann's Testing Statistical Hypothesis, the author writes Let $P_0$ and $P_1$ be probability distributions possessing densities $p_0$ and $p_1$ respectively w.r.t. a measure $\mu$ ...
61 views

What statistical test should I use? (GLMM and comparing 4 groups)

I'm analyzing the following case: I have a continuous response variable (named Vueltasmin), that allows positive, negative and null values, and factors as explanatory values (the explanatory variable ...
109 views

Pooled variance for post hoc tests for contingency table $\chi^{2}$ tests

The 2-by-2 contingency table test provides an alternate hypothesis testing approach to the $z$ test for proportion difference. Conveniently enough, a 2-by-k contingency table test provides an omnibus ...
4k views

Assessing A/B test results using Python

Problem's setting Assume having results of an A/B test. You let you users experience two variants of your website and you counted how many converted: ...
86 views

Determine p-value if $\Theta_0$ is not singleton

Here's what I've learned about general test problems: Let $X_1,\ldots X_n$ be random variables whose distributions depend on a parameter $\theta$ in some parameter space $\Theta.$ A general ...
89 views

Question on Inference - Catching Cheating Students

In their paper "Catching cheating students", Levitt and Lin propose a simple reduced-form method to identify cheating of students in exams. The strategy works as follows: For each possible pair of ...
123 views

Exact Null Distribution with Ties

I am interested in deriving exact null distributions for small-sample test statistics with non-trivial ties. Not fundamentally continuous variables that happen to have a few repeated values, but ...
473 views

Combining Z-scores using Stouffer's approach

Quick background: I require to obtain a single 'score' for comparison between experimental and simulated data. The parameter being tested is the height of various parts (num = 5) of the model. For the ...
459 views

Hypothesis testing for Gamma distribution

I have a sample $X_1,...,X_n \sim \Gamma(\alpha, \beta)$, where $\alpha, \beta$ - unknown parameters of Gamma distribution. How to build a test for testing $H_0:\alpha=1$ against $H_1:\alpha>1$?
67 views

GLMM for binomially distributed outcome, testing differential hypothesis

I am trying to find a way, to investigate differences between conditions in an experiment. The design is as follows: Depended Variable: Logical (answer is correct [correct accepted or correct ...
1k views

Z-Test Versus Chi-Square Test

I need to test whether response rate for Test B is larger than response Rate for Test A (i.e. AB Test). Null Hypothesis: is Response Rate (Test B) <> Response Rate (Test A). Alternative ...
115 views

Similarity test of binary time series data

I have two datasets of binary time series data and I would like to prove that these two datasets are similar. The datasets are coming form binary sensors installed in a smart home and they reflect ...
65 views

218 views

Testing Fixed and Random Effect of Mixed Model

This pdf illustrates nicely how is to test the random effect of multilevel model . But I am simulating data from a two-level model and estimating the parameters of the model for various combination of ...
148 views

Combinatorial Block Designs in Statistics

How are combinatorial designs such as BIBD and Steiner triple systems used in statistics in designs of experiments and hypothesis testing.
302 views

Compare sample means of normal distribution with autocorrelation issues

Please forgive me if this is a naive question, but I haven't been able to find an answer in my stats books or online. I'm working on a fish tracking dataset that consists of detections of tagged fish ...