Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

10
votes
0answers
919 views

Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
9
votes
0answers
1k views

When does a UMP test fail to exist?

I have a sample $X=(X_1, ...,X_n)\sim N(\mu,\sigma^2)$ with $\sigma^2$ known. The hypotheses are $H_0: \mu=\mu_0, H_1:\mu \neq \mu_0$. I know that in such a case an UMP test does not exist and so ...
9
votes
0answers
763 views

Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
7
votes
0answers
109 views

Is the Berry-Esseen theorem useful for justifying normality?

The Kolmogorov-Smirnov (KS) test tells one how confident they can be that a sample comes from a hypothesized distribution. It is my understanding that this test can be used to justify whether or not ...
6
votes
0answers
121 views

Compare two set of ratios

I'm looking for a statistical test to compare two set of ratios. For example A = [(2,13), (10,50), (23, 86), (33, 133)] B = [(6,10), (21,33), (30,62), (44,80)] ...
6
votes
0answers
145 views

Testing for Benford Law in real time

Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution ...
6
votes
0answers
514 views

Skew of p-value distribution under composite null hypotheses

On this page it says ...if HA holds, the p-values have a distribution for which values near 0 are more likely than values near 1. However the p-values may have a distribution that is not ...
5
votes
0answers
85 views

Exact Null Distribution with Ties

I am interested in deriving exact null distributions for small-sample test statistics with non-trivial ties. Not fundamentally continuous variables that happen to have a few repeated values, but ...
5
votes
0answers
45 views

Statistical test to see whether we drew overly-balanced samples

I have an urn with $R$ red and $B$ black balls. $R$ and $B$ are large numbers. I draw some $n$ samples of different sizes without replacement until the urn is empty. The sizes of the samples are not a ...
5
votes
0answers
649 views

Use Chi-Squared or Binomial Test if Distribution is not Known?

Suppose you have a set of data (eg. [a, b, a, a, b, b, etc.]), and you have the suspicion that the set of data follows the binomial distribution. Your Null Hypothesis is: The probability of success ...
5
votes
0answers
378 views

Hypothesis testing for binomial distribution: a simple real-life case

I'm refreshing (or maybe just acquiring) some stats skills, and I have a real-life situation, probably very simple since it's quite close to a typical example from stats courses, that I would like to ...
5
votes
0answers
163 views

Can you perform hypothesis testing on mutual information values?

I am working on a project that uses mutual information to determine how much information is shared between two "variables" x and y in two separate populations. Essentially, I have two groups and ...
5
votes
0answers
744 views

Proof of Kolmogorov-Smirnov test

Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the $...
5
votes
0answers
456 views

Hypothesis testing with non-iid observations

I have a dataset with observations which violate the independent and identically distributed (iid) assumption. The dataset is composed by a number of post-click conversion events (purchases on a ...
5
votes
0answers
172 views

Asymmetry of the Kullback-Leibler distance in hypothesis testing

My question is related to the asymmetry of the Kullback-Leibler distance. I'm using the discrete definition of the Kullback-Leibler distance, so we have: $$ KL(p,q) = \sum_{s \in S} p(s) \log\left( \...
5
votes
0answers
408 views

Testing for statistical differences of quantile regression line slopes

If I were to compare the statistical similarity between the slopes of OLS regression lines from two independent samples, I would use a t-test to test if the slopes are equal or not. I'd like to ...
5
votes
0answers
114 views

How to tell if a sample comes from a distribution with infinite mean?

I observe a sample from a distribution that I expect to be the hitting time $$\tau = \inf\{t>0| X(t)>a\}$$ where $X(t)$ is a Lévy process with $X(0)=0$ and $a$ is some constant. $X$ is not a ...
5
votes
0answers
142 views

Hypothesis test on the Euclidean length of an unknown vector

Question Suppose I observe a vector $\mathbf{x}=[X_1 \ldots X_n]$, where each $X_i=m_i+n_i$, with $n_i$ being an independent zero-mean Gaussian random variable with variance $\sigma^2$ (i.e. $n_i\sim\...
5
votes
0answers
119 views

Test for difference of distributions on a torus

I have two circular dependent variables and would like to test for a difference in the distributions (presumably circular means) between multiple treatment groups. There are a number of multivariate ...
5
votes
0answers
952 views

Binomial mid-p value

I've been under the impression that the mid-$p$ values generally control the Type I error, and consequently confidence intervals based on mid-$p$ values control the coverage. However I have checked ...
5
votes
0answers
731 views

Using the hypergeometric distribution for skipping events in transcriptome sequencing

My question is inspired by this post. However, its a bit more complicated than that to explain. I hope I succeed. I work with RNA-Seq data on alternative splicing in plants. For this discussion, lets ...
4
votes
0answers
74 views

Can I statistically describe a single case/outlier vs. a distribution?

I have a dataset consisting of body weight and corresponding age for a bunch of healthy subjects (grey triangles below). I fit a nonlinear function to this data and graphed a 95% prediction interval. ...
4
votes
0answers
62 views

A possible typo in the textbook?

On page 74 of Lehmann's Testing Statistical Hypothesis, the author writes Let $P_0$ and $P_1$ be probability distributions possessing densities $p_0$ and $p_1$ respectively w.r.t. a measure $\mu$ ...
4
votes
0answers
58 views

What statistical test should I use? (GLMM and comparing 4 groups)

I'm analyzing the following case: I have a continuous response variable (named Vueltasmin), that allows positive, negative and null values, and factors as explanatory values (the explanatory variable ...
4
votes
0answers
97 views

Pooled variance for post hoc tests for contingency table $\chi^{2}$ tests

The 2-by-2 contingency table test provides an alternate hypothesis testing approach to the $z$ test for proportion difference. Conveniently enough, a 2-by-k contingency table test provides an omnibus ...
4
votes
0answers
2k views

Assessing A/B test results using Python

Problem's setting Assume having results of an A/B test. You let you users experience two variants of your website and you counted how many converted: ...
4
votes
0answers
79 views

Determine p-value if $\Theta_0$ is not singleton

Here's what I've learned about general test problems: Let $X_1,\ldots X_n$ be random variables whose distributions depend on a parameter $\theta$ in some parameter space $\Theta.$ A general ...
4
votes
0answers
87 views

Question on Inference - Catching Cheating Students

In their paper "Catching cheating students", Levitt and Lin propose a simple reduced-form method to identify cheating of students in exams. The strategy works as follows: For each possible pair of ...
4
votes
0answers
379 views

Hypothesis testing for Gamma distribution

I have a sample $X_1,...,X_n \sim \Gamma(\alpha, \beta)$, where $\alpha, \beta$ - unknown parameters of Gamma distribution. How to build a test for testing $H_0:\alpha=1$ against $H_1:\alpha>1$?
4
votes
0answers
64 views

GLMM for binomially distributed outcome, testing differential hypothesis

I am trying to find a way, to investigate differences between conditions in an experiment. The design is as follows: Depended Variable: Logical (answer is correct [correct accepted or correct ...
4
votes
0answers
1k views

Z-Test Versus Chi-Square Test

I need to test whether response rate for Test B is larger than response Rate for Test A (i.e. AB Test). Null Hypothesis: is Response Rate (Test B) <> Response Rate (Test A). Alternative ...
4
votes
0answers
95 views

Similarity test of binary time series data

I have two datasets of binary time series data and I would like to prove that these two datasets are similar. The datasets are coming form binary sensors installed in a smart home and they reflect ...
4
votes
0answers
64 views

How to modify the f-test null hypothesis in ANOVA

Normally the null hypothesis for a f-test in ANOVA is $H_0:\mu_1=\mu_2=...=\mu_m$ for a comparison of $m$ groups. How do I alter the f-statistic so that the null hypothesis is $H_0:\mu_1=\mu_2=...=\...
4
votes
0answers
188 views

How to test if coefficients of two linear regressions differ in practice?

The Chow test is often suggested to test if the coefficients of two linear regressions differ or if a single linear regressions is more appropriate. However, Chow test assumes equality of the ...
4
votes
0answers
2k views

Zero-inflated model: non-finite value supplied by optim

So I have the following model predicting the presence of an animal on a certain spot. As a time unit quarter is initially used, but for one of the species of animals there is some (little) interesting ...
4
votes
0answers
247 views

What model contains Wilcoxon's rank sum test?

As Frank Harrel mentions in the answer to Logistic regression and Wilcoxon test, Wilcoxon's rank sum test (aka Mann-Whithey test) is contained in the proportional odds logistic regression, which would ...
4
votes
0answers
153 views

Fisher - Neyman dispute over weak and strong null hypotheses

I am trying to find information on one of the many exchanges between Fisher and (I believe, but cannot be sure) Neyman. I believe the exchange took place at one of the Royal Statistical Society ...
4
votes
0answers
2k views

Assumptions of correlation test vs regression slope test (significance testing)

If my understanding is correct, then the test on a regression slope in a simple bivariate regression - i.e. the test of $\mathcal{H}_0$: $b = 0$ in $Y' = a + bX$ and the test of a correlation, i.e. $...
4
votes
0answers
226 views

Is this popular approach to Bootstrap hypothesis testing correct?

A widely cited paper "Statistical Significance Tests for Machine Translation Evaluation" proposes usage of Bootstrap for evaluating significance of difference across systems for Machine Translation (...
4
votes
0answers
81 views

Testing differences between slopes by examining proportions rather than interactions

In a recent article published in Nature Genetics, Francioli et al. argue that the relationship between paternal age and mutation rate depends on what part of the genome you're looking at (intergenic ...
4
votes
0answers
760 views

In a permutation test, how to get a confidence interval for the estimated p-value?

Suppose that I have two groups of data of size $50$ and $51$. Assuming that I did an approximate Monte Carlo permutation test with 10,000 permutations randomly drawn, is there a way to find a ...
4
votes
0answers
188 views

Testing Fixed and Random Effect of Mixed Model

This pdf illustrates nicely how is to test the random effect of multilevel model . But I am simulating data from a two-level model and estimating the parameters of the model for various combination of ...
4
votes
0answers
138 views

Combinatorial Block Designs in Statistics

How are combinatorial designs such as BIBD and Steiner triple systems used in statistics in designs of experiments and hypothesis testing.
4
votes
0answers
300 views

Compare sample means of normal distribution with autocorrelation issues

Please forgive me if this is a naive question, but I haven't been able to find an answer in my stats books or online. I'm working on a fish tracking dataset that consists of detections of tagged fish ...
4
votes
0answers
108 views

Determining statistical significance in difference between two CRF models

I have trained two Conditional Random Field (CRF) models Ma and Mb in CRF++, the first using a simple set of features, the ...
4
votes
0answers
320 views

appropriate hypothesis test for non-negative quantity

This question is intentionally written in a very abstract way, because I am interested in a generic solution, or for pointers to different possible concrete cases in different fields: I am measuring ...
4
votes
0answers
653 views

Chi-squared test and binomial distribution

In some circumstances the Chi-square test can be replaced by a test based on the binomial distribution. The common issue with using the binomial distribution is that it may quickly become tricky to ...
4
votes
0answers
342 views

How Should I Find Statistically Significant Differences Between Two Markov Models?

Suppose I have N Markov Models of M states representing the behaviour patterns of 2 different groups (note: fully observable models, no hidden states), and have stored each model as a matrix of ...
4
votes
0answers
2k views

Neyman-Pearson lemma: critical region and hypothesis testing

Let $X_1,X_2,...,X_n$ be i.i.d r.v's with common p.d.f. $$ \mbox f(x)=\frac{x^5e^{-x/\theta}}{5!\theta^6} $$ where $\theta$ > 0. Show that the Neyman-Pearson lemma produces a test of $H_0: \...
4
votes
0answers
109 views

Comparing 2 independent counts of the same data

I am looking for an appropriate statistical method to test my hypothesis that there is no significant difference between counts conducted by 2 individuals of the same data. The data is as follows: I ...