Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

2
votes
2answers
187 views

Based on my data, is Jack likely to be clumsy?

I posted this on mathoverflow, but they sent me here. This question relates to a problem I had at work a while ago, doing a little data mining at a car rental company. Names changed, of course. I'm ...
12
votes
1answer
2k views

Testing two independent samples for null of same skew?

What tests are available for testing two independent samples for the null hypothesis that they come from populations with the same skew? There is a classical 1-sample test for whether the skew equals ...
7
votes
2answers
574 views

Hypothesis testing that one time-series of a measure of entropy doesn't belong to a population

Disclaimer: I'm a software engineer, not a statistician, so please forgive any blunt error :-) I have a set of time-series "curves", each measuring the entropy of a given artifact. Now, I'm standing ...
9
votes
2answers
748 views

Comparing two genetic algorithms

I have two implementations of a genetic algorithm which are supposed to behave equivalently. However due to technical restrictions which cannot be resolved their output is not exactly the same, given ...
10
votes
2answers
2k views

What is a test of independence?

I need to define what a test of independence is, without the use of heavily statistic terms.
18
votes
6answers
15k views

Appropriate normality tests for small samples

So far, I've been using the Shapiro-Wilk statistic in order to test normality assumptions in small samples. Could you please recommend another technique?
3
votes
1answer
450 views

Comparing 2 independent non-central t statistics

One estimate of the 'quality' of a portfolio of stocks is the Sharpe ratio, which is defined as the mean of the returns divided by the standard deviation of the returns (modulo adjustments for risk ...
15
votes
2answers
2k views

Why does the supremum of the Brownian bridge have the Kolmogorov–Smirnov distribution?

The Kolmogorov–Smirnov distribution is known from the Kolmogorov–Smirnov test. However, it is also the distribution of the supremum of the Brownian bridge. Since this is far from obvious (to me), I ...
6
votes
1answer
8k views

On univariate outlier tests (or: Dixon Q versus Grubbs)

In (most of) the analytical chemistry literature, the standard test for detecting outliers in univariate data (e.g. a sequence of measurements of some parameter) is Dixon's Q test. Invariably, all the ...
3
votes
1answer
290 views

What resources/methods exist for testing/validation or evaluation of Statistical Methods

I'm interested in the process of testing or validating a particular implementation of a statistical method, and what datasets and/or published analysis exist that could be used to do this in practice. ...
3
votes
2answers
954 views

Testing implementation of Anderson-Darling test for uniform RV

I am trying to write unit tests for a whole mess of statistics code. Some of the unit tests take the form: generate a sample following a null hypothesis, use code to get a p-value under that null, ...
42
votes
5answers
6k views

Why is multiple comparison a problem?

I find it hard to understand what really is the issue with multiple comparisons. With a simple analogy, it is said that a person who will make many decisions will make many mistakes. So very ...
6
votes
2answers
13k views

Statistical test for difference between two odds ratios?

I am attempting to compare two diagnostic odds ratios (DORs). I would like to know of a statistical test which will allow me to do this. Please help! Thank you!
15
votes
3answers
7k views

Robust t-test for mean

I am trying to test the null $E[X] = 0$, against the local alternative $E[X] > 0$, for a random variable $X$, subject to mild to medium skew and kurtosis of the random variable. Following ...
2
votes
1answer
174 views

Heterogeneity with two studies

With data from two centres I want to account for potential heterogeneity or confounders between two centers. So the analysis will initially be stratified by clinical center and a chi square test ...
38
votes
8answers
49k views

How can I test if given samples are taken from a Poisson distribution?

I know of normality tests, but how do I test for "Poisson-ness"? I have sample of ~1000 non-negative integers, which I suspect are taken from a Poisson distribution, and I would like to test that.
26
votes
3answers
17k views

Is Kolmogorov-Smirnov test valid with discrete distributions?

I'm comparing a sample and checking whether it distributes as some, discrete, distribution. However, I'm not enterily sure that Kolmogorov-Smirnov applies. Wikipedia seems to imply it does not. If it ...
6
votes
7answers
805 views

Determine if three is statistically different than ten for a very large number of observations (1,000,000)

For 1,000,000 observations, I observed a discrete event, X, 3 times for the control group and 10 times for the test group. How do I determine for a large number of observations (1,000,000), if three ...
5
votes
1answer
3k views

Test if probabilities are statistically different?

I completed a Monte Carlo simulation that consisted of one million ($10^6$) individual simulations. The simulation returns a variable, $p$, that can be either 1 or 0. I then weight the simulations ...
49
votes
5answers
30k views

Correct spelling (capitalization, italicization, hyphenation) of “p-value”?

I realize this is pedantic and trite, but as a researcher in a field outside of statistics, with limited formal education in statistics, I always wonder if I'm writing "p-value" correctly. ...
21
votes
1answer
26k views

Multiple hypothesis testing correction with Benjamini-Hochberg, p-values or q-values?

Given a list of p-values generated from independent tests, sorted in ascending order, one can use the Benjamini-Hochberg procedure for multiple testing correction. For each p-value, the Benjamini-...
11
votes
3answers
4k views

Fisher's Exact Test with weights?

Does anyone know of a variation of Fisher's Exact Test which takes weights into account? For instance sampling weights. So instead of the usual 2x2 cross table, every data point has a "mass" or "size" ...
15
votes
5answers
4k views

Comparing the variance of paired observations

I have $N$ paired observations ($X_i$, $Y_i$) drawn from a common unknown distribution, which has finite first and second moments, and is symmetric around the mean. Let $\sigma_X$ the standard ...
9
votes
6answers
8k views

What ways are there to show two analytical methods are equivalent?

I have two different analytical methods that can measure the concentration of a particular molecule in a matrix (for instance measure the amount of salt in water) The two methods are different, and ...
42
votes
6answers
9k views

Motivation for Kolmogorov distance between distributions

There are many ways to measure how similar two probability distributions are. Among methods which are popular (in different circles) are: the Kolmogorov distance: the sup-distance between the ...
10
votes
2answers
793 views

The trinity of tests in maximum likelihood: what to do when faced with contradicting conclusions?

The Wald, Likelihood Ratio and Lagrange Multiplier tests in the context of maximum likelihood estimation are asymptotically equivalent. However, for small samples, they tend to diverge quite a bit, ...
7
votes
3answers
1k views

What do you think is the best goodness of fit test?

I am looking at fitting distributions to data (with a particular focus on the tail) and am leaning towards Anderson-Darling tests rather than Kolmogorov-Smirnov. What do you think are the relative ...
228
votes
14answers
437k views

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
13
votes
8answers
2k views

Testing random variate generation algorithms

Which methods are used for testing random variate generation algorithms?