Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

2,459 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
13
votes
0answers
136 views

Comparison of statistical tests exploring co-dependence of two binary variables

Suppose we observe data $(X_i,Y_i)_{i=1,...,n}$ on two binary variables: $X\in\{0,1\}$ and $Y\in\{0,1\}$. We would like to test if $X$ and $Y$ are co-dependent (related). Standard suggestions in ...
13
votes
0answers
1k views

How to compare of two Spearman correlations matrices?

I have two non-parametric rank correlations matrices emp and sim (for example, based on Spearman's $\rho$ rank correlation ...
13
votes
1answer
2k views

When does a UMP test fail to exist?

I have a sample $X=(X_1, ...,X_n)\sim N(\mu,\sigma^2)$ with $\sigma^2$ known. The hypotheses are $H_0: \mu=\mu_0, H_1:\mu \neq \mu_0$. I know that in such a case an UMP test does not exist and so ...
11
votes
0answers
984 views

Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
9
votes
0answers
861 views

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
8
votes
1answer
1k views

How to handle multiple measurements per participant, with categorical data?

I've done an experiment where I've collected measurements from a number of participants. Each relevant data point has two variables, both categorical: in fact, each variable has two possible values (...
7
votes
0answers
153 views

Is the Berry-Esseen theorem useful for justifying normality?

The Kolmogorov-Smirnov (KS) test tells one how confident they can be that a sample comes from a hypothesized distribution. It is my understanding that this test can be used to justify whether or not ...
7
votes
1answer
1k views

How to test for the difference in skewness of two samples?

I have two samples. From looking at their densities, one appears symmetrical and the other from some right-tailed distribution. I would like to test that the two do not have the same skewness (...
7
votes
1answer
218 views

Testing for Benford Law in real time

Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution ...
6
votes
0answers
486 views

Testing for statistical differences of quantile regression line slopes

If I were to compare the statistical similarity between the slopes of OLS regression lines from two independent samples, I would use a t-test to test if the slopes are equal or not. I'd like to ...
6
votes
0answers
573 views

Skew of p-value distribution under composite null hypotheses

On this page it says ...if HA holds, the p-values have a distribution for which values near 0 are more likely than values near 1. However the p-values may have a distribution that is not ...
6
votes
1answer
109 views

Unit testing sampling methods

I'm writing a bit of code that makes pretty heavy use of sampling (eg, MCMC, Particle Filters, etc), and I would really like to test it to make sure that it's doing what I think it is before claiming ...
6
votes
1answer
694 views

How do I compare date-ranges from a time series?

I have a time series which contains monthly readings for air pollution in a city. The seasonality from this time series has been removed. Given two date ranges, for example Jan-Aug 2008 and Jan-Aug ...
5
votes
0answers
128 views

Probability of selecting maximum in bivariate correlated order statistics?

In a testing, ranking, or selection scenario, we have samples of size n where a measurement is correlated 0<r<1 with some second variable of interest; they are bivariate normally distributed. We'...
5
votes
1answer
159 views

Why there's never a good reason to use the Jarque-Bera test

This question was inspired by a comment made by @whuber in response to this question regarding the use of distributional tests. The comment stated there is never any reason to use [the Jarque-Bera ...
5
votes
0answers
5k views

Assessing A/B test results using Python

Problem's setting Assume having results of an A/B test. You let you users experience two variants of your website and you counted how many converted: ...
5
votes
0answers
93 views

Question on Inference - Catching Cheating Students

In their paper "Catching cheating students", Levitt and Lin propose a simple reduced-form method to identify cheating of students in exams. The strategy works as follows: For each possible pair of ...
5
votes
0answers
139 views

Exact Null Distribution with Ties

I am interested in deriving exact null distributions for small-sample test statistics with non-trivial ties. Not fundamentally continuous variables that happen to have a few repeated values, but ...
5
votes
0answers
541 views

Combining Z-scores using Stouffer's approach

Quick background: I require to obtain a single 'score' for comparison between experimental and simulated data. The parameter being tested is the height of various parts (num = 5) of the model. For the ...
5
votes
0answers
50 views

Statistical test to see whether we drew overly-balanced samples

I have an urn with $R$ red and $B$ black balls. $R$ and $B$ are large numbers. I draw some $n$ samples of different sizes without replacement until the urn is empty. The sizes of the samples are not a ...
5
votes
0answers
500 views

Hypothesis testing for Gamma distribution

I have a sample $X_1,...,X_n \sim \Gamma(\alpha, \beta)$, where $\alpha, \beta$ - unknown parameters of Gamma distribution. How to build a test for testing $H_0:\alpha=1$ against $H_1:\alpha>1$?
5
votes
0answers
267 views

Cross model comparison of quantile regression coefficients

I am looking for a way to compare coefficients obtained from quantile regression. The two surveyed models are nested, estimated on the same sample and for the same quantile. $$ Y = \beta_1X+\epsilon_2\...
5
votes
1answer
904 views

2-Sample Proportions z-test vs Fisher's Exact Test

I want to evaluate the difference in proportions in declined applications and exceptions from policy from one quarter of data to the next. My sample sizes are generally not small (several hundred to ...
5
votes
0answers
1k views

Use Chi-Squared or Binomial Test if Distribution is not Known?

Suppose you have a set of data (eg. [a, b, a, a, b, b, etc.]), and you have the suspicion that the set of data follows the binomial distribution. Your Null Hypothesis is: The probability of success ...
5
votes
0answers
414 views

Hypothesis testing for binomial distribution: a simple real-life case

I'm refreshing (or maybe just acquiring) some stats skills, and I have a real-life situation, probably very simple since it's quite close to a typical example from stats courses, that I would like to ...
5
votes
1answer
770 views

Multivariate differences between groups controlling for one factor (MANOVA)

I have a sample of 100 participants who have scores on 5 different variables (V1-V5). Some participants took part in a workshop, others did not. I am interested in investigating the influence of ...
5
votes
0answers
2k views

Assumptions of correlation test vs regression slope test (significance testing)

If my understanding is correct, then the test on a regression slope in a simple bivariate regression - i.e. the test of $\mathcal{H}_0$: $b = 0$ in $Y' = a + bX$ and the test of a correlation, i.e. $...
5
votes
0answers
588 views

Confidence-interval / p-value duality vs. frequentist interpretation of CIs

Many sources suggest that there is a duality between confidence intervals and hypothesis testing.(*) But I'm having trouble making sense of this philosophically. The frequentist interpretation of a ...
5
votes
0answers
2k views

Testing for conditional independence: What's the correct way?

My goal is to check if two variables $X$ and $Y$ are conditionally independent given $Z$. For simplicity, let's assume the joint distribution is multivariate normal. In this case, we can compute ...
5
votes
1answer
79 views

Testing a proportion in an online setting

I work in an online security setting. My goal is to detect if the number of locked accounts per time unit is stable or not. I've tried several approaches, detailed below, but I am not satisfied yet. ...
5
votes
0answers
851 views

Proof of Kolmogorov-Smirnov test

Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the $...
5
votes
0answers
602 views

Hypothesis testing with non-iid observations

I have a dataset with observations which violate the independent and identically distributed (iid) assumption. The dataset is composed by a number of post-click conversion events (purchases on a ...
5
votes
0answers
231 views

Asymmetry of the Kullback-Leibler distance in hypothesis testing

My question is related to the asymmetry of the Kullback-Leibler distance. I'm using the discrete definition of the Kullback-Leibler distance, so we have: $$ KL(p,q) = \sum_{s \in S} p(s) \log\left( \...
5
votes
0answers
120 views

How to tell if a sample comes from a distribution with infinite mean?

I observe a sample from a distribution that I expect to be the hitting time $$\tau = \inf\{t>0| X(t)>a\}$$ where $X(t)$ is a Lévy process with $X(0)=0$ and $a$ is some constant. $X$ is not a ...
5
votes
0answers
147 views

Hypothesis test on the Euclidean length of an unknown vector

Question Suppose I observe a vector $\mathbf{x}=[X_1 \ldots X_n]$, where each $X_i=m_i+n_i$, with $n_i$ being an independent zero-mean Gaussian random variable with variance $\sigma^2$ (i.e. $n_i\sim\...
5
votes
0answers
126 views

Test for difference of distributions on a torus

I have two circular dependent variables and would like to test for a difference in the distributions (presumably circular means) between multiple treatment groups. There are a number of multivariate ...
5
votes
0answers
1k views

Binomial mid-p value

I've been under the impression that the mid-$p$ values generally control the Type I error, and consequently confidence intervals based on mid-$p$ values control the coverage. However I have checked ...
5
votes
0answers
758 views

Using the hypergeometric distribution for skipping events in transcriptome sequencing

My question is inspired by this post. However, its a bit more complicated than that to explain. I hope I succeed. I work with RNA-Seq data on alternative splicing in plants. For this discussion, lets ...
4
votes
1answer
54 views

Correlation between logrank (log-rank) test statistics with common control

Say I have a $K$ arm experiment that generates survival (time-to-event) endpoints. There are $K-1$ experimental arms and a single control arm. Say I compute a log rank test statistic comparing the ...
4
votes
1answer
58 views

Confusion about Anderson-Darling test

The Anderson-Darling test statistic is $$ A^2 := n \int_{-\infty}^{\infty} \frac{(F_n(x)-F(x))^2}{F(x)(1-F(x))} \, \mathrm{d}F(x) $$ We know that the samples $$ Z(x) := \frac{\sqrt{n}(F_{n}(x) - F(x))}...
4
votes
1answer
78 views

Why is the population standard deviation approximated as the sample standard deviation?

This question addresses calculating a p value from the mean and standard deviation statistics of a sample. I understand that the -general- philosophy is to divide the sample standard deviation by the ...
4
votes
0answers
145 views

Test for Lipschitz continuity (is there some?)

Let $x_1, \dots, x_n$ be a random sample from a distribution $D$. Say, I want to test whether $F(z)$, the cdf of $D$, is Lipschitz continuous, i.e. there exists $L$ such that $F(z + \delta) - F(z) \...
4
votes
0answers
57 views

How is it possible to control false discovery rate (FDR) without knowing the power and the prevalence of the nulls?

If we have p-value 0.05, to calculate probability of our discovery to be false positive, we need to use complex formula with prevalence (prior probability) and statistical power. If we have lots of ...
4
votes
0answers
133 views

Comparing multiple frequency distributions from data and from simulation

I am attempting to analyze the shuffling in the card based online game Magic Arena, which I suspect has a specific bug that causes the distribution of outputs to be biased. Is the approach I have ...
4
votes
1answer
78 views

Can you statistically compare one subset of a population with the whole population?

I have data (total $n=90$) from a district of testing scores from $3$ subsequent years. I would like to compare MY school to the other schools, but I do not know which schools the other participants ...
4
votes
0answers
80 views

P-value: Fisherian vs. contemporary frequentist definitions

I am trying to see if I understand the definition of $p$-value as used by Sir R. A. Fisher and the one used today by frequentist statisticians (not sure how to call it better). $p$-value according ...
4
votes
0answers
80 views

Can I statistically describe a single case/outlier vs. a distribution?

I have a dataset consisting of body weight and corresponding age for a bunch of healthy subjects (grey triangles below). I fit a nonlinear function to this data and graphed a 95% prediction interval. ...
4
votes
1answer
64 views

How to test whether 2 prediction intervals are statistically different?

I've been struggling with this for a while now, hopefully someone will know how to help me :) Here it is : 1) I'm using a linear mixed effects model on longitudinal data (biological values of many ...
4
votes
0answers
229 views

The Mediterranean diet - statistics issues with the updated paper

In 2013, a group of researchers published a paper with results on a randomized trial of the Mediterranean diet, finding that it appears to have significant health benefits. Today, they retracted ...
4
votes
0answers
66 views

A possible typo in the textbook?

On page 74 of Lehmann's Testing Statistical Hypothesis, the author writes Let $P_0$ and $P_1$ be probability distributions possessing densities $p_0$ and $p_1$ respectively w.r.t. a measure $\mu$ ...