Are there any methods to make parametric hypothesis testing assuming that data is sampled from a known but non-normal continuous distribution?

I'm glad to see a solution to any particular distribution. I'll be happy if there is some kind of cookbook or bunch of scientific papers for different distributions.

P.S. It's more for theoretical curiosity's sake rather than practical purposes.

  • 1
    $\begingroup$ See if generalized linear models get at what you want to do. $\endgroup$ – Dave Jun 1 at 15:13
  • 1
    $\begingroup$ What would you be interested in testing? Difference between means of groups in which the response is not normal? $\endgroup$ – Demetri Pananos Jun 1 at 15:16
  • $\begingroup$ @DemetriPananos yes, difference between means, and means itself for one-sample test. $\endgroup$ – True do day Jun 1 at 15:22

With enough data (whatever "enough" means), a t-test will suffice. Briefly, the central limit theorem says that the sampling distribution of the sample mean is normal with mean $\mu$ and variance $\sigma^2$. Because we have to estimate $\sigma$, that means we can use the t test to test for a difference in means.

The story is different when we don't have "enough" data. As Dave mentions, generalized linear models allow us to test for differences in means between to groups with additional restrictions on the likelihood (that is, the distribution of the data. Namely, the data must belong to the exponential family e.g. Normal, Poisson, Binomial, Gamma, with possible extensions like the beta binomial, negative binomial, and so on). In this case, we can fit a GLM using a binary indicator or group assignment (e.g. 1 for test, 0 for control). The significance of the difference between groups is determined by the Wald test for the associated coefficient, with the estimate of the difference depending on what the link function is. We can even use GLM to estimate the mean if a single group.

| cite | improve this answer | |
  • 1
    $\begingroup$ If we have two groups that have Poisson (for instance) distributions and want to compare their means, we can run them through the t-test and get some inference about the means. How would this compare to doing a Poisson GLM on the indicator variable? $\endgroup$ – Dave Jun 2 at 16:52
  • $\begingroup$ @Dave The inference with respect to the null hypothesis may be the same, but the inference as to differences consistent with the data may be different. It is not hard to simulate data in which differences in means between independent poisson samples is correctly identified, but the associated confidence interval yields differences which would squarely place one rate parameter less than 0. The Poisson GLM acts on the multiplicative scale, which sidesteps this. $\endgroup$ – Demetri Pananos Jun 2 at 23:24

Given a particular parametric assumption, and a suitable test statistic, if you can compute the distribution of the test statistic under the null, you can perform a hypothesis test.

(Simulation could be used to compute p-values where such a calculation is not tractable/convenient.)

The main issue is then "Given a parametric distributional assumption, how do we find a good test statistic?". This boils down to finding test statistics that have good power under that parametric assumption.

This task - and the tests that result - is one focus of statistical theory, and many books used in statistics degree programs discuss it in detail.

Some useful Wikipedia links:

Likelihood-ratio test

Score test

Wald test

| cite | improve this answer | |
  • $\begingroup$ I understand it's not a trivial task, like using a t-test. Can I get links to the books you mentioned? $\endgroup$ – True do day Jun 2 at 15:17
  • 1
    $\begingroup$ In nice cases it can sometimes be relatively simple (cases often set as exercises); for example with a uniform on $(0,\theta)$, the LRT for a one-sided, one sample test has simple form. I can't link all the suitable books but I could mention a few example references; a book like Wackerly et al, Mathematical Statistics with Applications has some basic details (needing little more than basic calculus); going up a bit there's Wasserman's rather grandly titled All of Statistics; Casella and Berger's Statistical Inference has considerably more detail (at a somewhat more technical level) $\endgroup$ – Glen_b Jun 3 at 8:03
  • $\begingroup$ There are many other books at a variety of levels. $\endgroup$ – Glen_b Jun 3 at 8:03
  • 1
    $\begingroup$ Incidentally, it's not that hard to come up with examples where the t-test will not suffice -- either because the significance level is too far from correct, or the power is very poor, or both. $\endgroup$ – Glen_b Jun 7 at 5:08

You can check it numerically from the following Mathematica code:

data = RandomVariate[WeibullDistribution[3.5, 2], 50]

{1.46493, 1.60199, 2.41101, 1.64718, 1.41811, 1.51449, 1.65843, \ 1.07234, 2.09288, 1.51687, 1.89899, 2.34875, 2.46311, 2.53133, \ 2.03461, 2.31118, 2.77079, 2.33359, 2.20265, 1.19708, 1.61877, \ 1.27819, 2.01383, 2.54109, 1.2091, 1.59319, 1.36364, 1.75678, \ 0.751303, 2.86823, 1.62412, 1.90921, 1.61494, 0.971562, 0.976829, \ 2.26365, 1.69119, 1.37706, 2.39278, 1.94787, 2.27209, 2.80593, \ 2.38035, 1.08103, 1.15813, 1.07912, 2.31321, 1.78985, 2.63854, \ 0.343844}

pars = FindDistributionParameters[data, WeibullDistribution[a, b]]

{a -> 3.49419, b -> 2.00449}

`h = DistributionFitTest[data, WeibullDistribution[a, b], 

h["TestDataTable", All]`

\begin{array}{l|ll} \text{} & \text{Statistic} & \text{P-Value} \\ \hline \text{Anderson-Darling} & 0.293282 & 0.632596 \\ \text{Cram{\' e}r-von Mises} & 0.0424133 & 0.629623 \\ \text{Kolmogorov-Smirnov} & 0.0687611 & 0.798839 \\ \text{Kuiper} & 0.114113 & 0.675256 \\ \text{Pearson }\chi ^2 & 6.4 & 0.493895 \\ \text{Watson }U^2 & 0.039617 & 0.661233 \\ \end{array}

-2 LogLikelihood[WeibullDistribution[a, b], data] /. pars

(You can Find AIC etc.)


    p1 = Histogram[data, 7, "PDF"];
p2 = Plot[PDF[WeibullDistribution[a, b] /. pars, x], {x, 0, 3.5}];
Show[p1, p2]

enter image description here

If you change simulated data with real data you can test it with any non-normal distribution.


    n = 50; r = 100;
data = RandomVariate[WeibullDistribution[3.5, 2], {r, n}];
pars = Table[{a, b} /. 
     WeibullDistribution[a, b]], {i, 1, r}];

{3.55039, 2.0105}

se = StandardDeviation[pars]/Sqrt[n]

{0.0519725, 0.0124021}

In that way you can contract Confidence Intervals.

| cite | improve this answer | |
  • $\begingroup$ Is there any way to get confidence intervals (for example, for means) using methods like this? I've only come to vary the distribution parameters until I get a small enough p-value. Then, from these "boundary" parameters to get the interval boundaries. Is there more "parametric" way? $\endgroup$ – True do day Jun 2 at 19:14
  • $\begingroup$ @Truedoday see edit-1 $\endgroup$ – SAAN Jun 3 at 6:19
  • $\begingroup$ If I calculate confidence interval, I need to take into account significance level $\alpha$ given, hence p-values from DistributionFitTest too, independent of the std.dev of pars. Did I get it wrong? $\endgroup$ – True do day Jun 3 at 7:56
  • $\begingroup$ @Truedoday DistributionFitTest did not find the confidence interval p-values. After construing confidence interval you can check each estimate lie between interval or not. From this ratio(lie/total) you can find "power of test". $\endgroup$ – SAAN Jun 3 at 8:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.