# Parametric hypothesis testing for non-normal data

Are there any methods to make parametric hypothesis testing assuming that data is sampled from a known but non-normal continuous distribution?

I'm glad to see a solution to any particular distribution. I'll be happy if there is some kind of cookbook or bunch of scientific papers for different distributions.

P.S. It's more for theoretical curiosity's sake rather than practical purposes.

• See if generalized linear models get at what you want to do.
– Dave
Commented Jun 1, 2020 at 15:13
• What would you be interested in testing? Difference between means of groups in which the response is not normal? Commented Jun 1, 2020 at 15:16
• @DemetriPananos yes, difference between means, and means itself for one-sample test. Commented Jun 1, 2020 at 15:22

With enough data (whatever "enough" means), a t-test will suffice. Briefly, the central limit theorem says that the sampling distribution of the sample mean is normal with mean $$\mu$$ and variance $$\sigma^2$$. Because we have to estimate $$\sigma$$, that means we can use the t test to test for a difference in means.

The story is different when we don't have "enough" data. As Dave mentions, generalized linear models allow us to test for differences in means between to groups with additional restrictions on the likelihood (that is, the distribution of the data. Namely, the data must belong to the exponential family e.g. Normal, Poisson, Binomial, Gamma, with possible extensions like the beta binomial, negative binomial, and so on). In this case, we can fit a GLM using a binary indicator or group assignment (e.g. 1 for test, 0 for control). The significance of the difference between groups is determined by the Wald test for the associated coefficient, with the estimate of the difference depending on what the link function is. We can even use GLM to estimate the mean if a single group.

• If we have two groups that have Poisson (for instance) distributions and want to compare their means, we can run them through the t-test and get some inference about the means. How would this compare to doing a Poisson GLM on the indicator variable?
– Dave
Commented Jun 2, 2020 at 16:52
• @Dave The inference with respect to the null hypothesis may be the same, but the inference as to differences consistent with the data may be different. It is not hard to simulate data in which differences in means between independent poisson samples is correctly identified, but the associated confidence interval yields differences which would squarely place one rate parameter less than 0. The Poisson GLM acts on the multiplicative scale, which sidesteps this. Commented Jun 2, 2020 at 23:24

Given a particular parametric assumption, and a suitable test statistic, if you can compute the distribution of the test statistic under the null, you can perform a hypothesis test.

(Simulation could be used to compute p-values where such a calculation is not tractable/convenient.)

The main issue is then "Given a parametric distributional assumption, how do we find a good test statistic?". This boils down to finding test statistics that have good power under that parametric assumption.

This task - and the tests that result - is one focus of statistical theory, and many books used in statistics degree programs discuss it in detail.

Score test

Wald test

• I understand it's not a trivial task, like using a t-test. Can I get links to the books you mentioned? Commented Jun 2, 2020 at 15:17
• In nice cases it can sometimes be relatively simple (cases often set as exercises); for example with a uniform on $(0,\theta)$, the LRT for a one-sided, one sample test has simple form. I can't link all the suitable books but I could mention a few example references; a book like Wackerly et al, Mathematical Statistics with Applications has some basic details (needing little more than basic calculus); going up a bit there's Wasserman's rather grandly titled All of Statistics; Casella and Berger's Statistical Inference has considerably more detail (at a somewhat more technical level) Commented Jun 3, 2020 at 8:03
• There are many other books at a variety of levels. Commented Jun 3, 2020 at 8:03
• Incidentally, it's not that hard to come up with examples where the t-test will not suffice -- either because the significance level is too far from correct, or the power is very poor, or both. Commented Jun 7, 2020 at 5:08

You can check it numerically from the following Mathematica code:

data = RandomVariate[WeibullDistribution[3.5, 2], 50]


{1.46493, 1.60199, 2.41101, 1.64718, 1.41811, 1.51449, 1.65843, \ 1.07234, 2.09288, 1.51687, 1.89899, 2.34875, 2.46311, 2.53133, \ 2.03461, 2.31118, 2.77079, 2.33359, 2.20265, 1.19708, 1.61877, \ 1.27819, 2.01383, 2.54109, 1.2091, 1.59319, 1.36364, 1.75678, \ 0.751303, 2.86823, 1.62412, 1.90921, 1.61494, 0.971562, 0.976829, \ 2.26365, 1.69119, 1.37706, 2.39278, 1.94787, 2.27209, 2.80593, \ 2.38035, 1.08103, 1.15813, 1.07912, 2.31321, 1.78985, 2.63854, \ 0.343844}

pars = FindDistributionParameters[data, WeibullDistribution[a, b]]


{a -> 3.49419, b -> 2.00449}

h = DistributionFitTest[data, WeibullDistribution[a, b],
"HypothesisTestData"]

h["TestDataTable", All]


$$\begin{array}{l|ll} \text{} & \text{Statistic} & \text{P-Value} \\ \hline \text{Anderson-Darling} & 0.293282 & 0.632596 \\ \text{Cram{\' e}r-von Mises} & 0.0424133 & 0.629623 \\ \text{Kolmogorov-Smirnov} & 0.0687611 & 0.798839 \\ \text{Kuiper} & 0.114113 & 0.675256 \\ \text{Pearson }\chi ^2 & 6.4 & 0.493895 \\ \text{Watson }U^2 & 0.039617 & 0.661233 \\ \end{array}$$

-2 LogLikelihood[WeibullDistribution[a, b], data] /. pars


(You can Find AIC etc.)

82.6564

    p1 = Histogram[data, 7, "PDF"];
p2 = Plot[PDF[WeibullDistribution[a, b] /. pars, x], {x, 0, 3.5}];
Show[p1, p2]


If you change simulated data with real data you can test it with any non-normal distribution.

Edit-1

    n = 50; r = 100;
data = RandomVariate[WeibullDistribution[3.5, 2], {r, n}];
pars = Table[{a, b} /.
FindDistributionParameters[data[[i]],
WeibullDistribution[a, b]], {i, 1, r}];
Mean[pars]


{3.55039, 2.0105}

se = StandardDeviation[pars]/Sqrt[n]


{0.0519725, 0.0124021}

In that way you can contract Confidence Intervals.

• Is there any way to get confidence intervals (for example, for means) using methods like this? I've only come to vary the distribution parameters until I get a small enough p-value. Then, from these "boundary" parameters to get the interval boundaries. Is there more "parametric" way? Commented Jun 2, 2020 at 19:14
• @Truedoday see edit-1
– SAAN
Commented Jun 3, 2020 at 6:19
• If I calculate confidence interval, I need to take into account significance level $\alpha$ given, hence p-values from DistributionFitTest too, independent of the std.dev of pars. Did I get it wrong? Commented Jun 3, 2020 at 7:56
• @Truedoday DistributionFitTest did not find the confidence interval p-values. After construing confidence interval you can check each estimate lie between interval or not. From this ratio(lie/total) you can find "power of test".
– SAAN
Commented Jun 3, 2020 at 8:08