Lets say I have a p value that is calculated from a population of standard deviations; and I have another p value that is calculated from a population of skewness. Can I combine these two p values together using Fisher's method? (it requires the two p values to be independent.)

  • 1
    $\begingroup$ How are these populations related, if at all? Do they appear to come from, or are they assumed to come from, some specific distribution family such as a Normal distribution? $\endgroup$ – whuber Nov 4 '16 at 23:21

It's not possible to say from the information supplied.

If your skewness and standard deviation are on the same sample, the dependence or independence of sample skewness and sample standard deviation depends on the distribution you're sampling from. (In general they will be dependent; in some particular cases they might happen to be independent).

A quick few simulations suggests that the two must be at least very nearly independent in the case of normality -- even at small $n$ (where I'd expect dependence to be more obvious), the conditional distribution of the skewness seems to be essentially the same for large and small values of the sample standard deviation for samples from a normal distribution:

Conditional kernel density estimates of sample skewness for large and small sample standard deviation, samples from normal distribution, n=7

Similar results were obtained at larger sample sizes (n=15) and slicing the other way (holding sample skewness almost-constant and looking at the density of sample standard deviation). Similarly, bivariate plots of the cube root of the variance (to get near-symmetry) against the sample skewness showed no obvious sign of dependence:

Plot of sample sd (to the 2/3 power) vs sample skewness, normal samples, n=7, 100,000 simulations. Appearance is rather like bivariate normal under independence

[I imagine there would be some result relating to it for the normal case; I'll try to take a look at Kendall and Stuart next time I am near a copy and see if they mention it.]

By contrast, sampling from some other distributions shows quite clear dependence. Here's an example with uniform sampling:

Plot of sample sd (to the 2/3 power) vs sample skewness, uniform samples, n=7, 100,000 simulations. There's distinct "scalloping" around the top edge (i.e. at larger standard deviations), giving it a shape like the top of a crab-shell

If you're calculating the two statistics on two independent samples they should be independent, however.

Combining p-values from dependent samples: One can still combine p-values if you have a good idea of their joint distribution under the null hypotheses. If you simulate many samples from some distributional model and calculate the pair of p-values, you can see approximately what the joint distribution of p-values looks like and choose some combined measure of extremeness. The Fisher method should work well if they're not too dependent (even if not nearly independent enough to use the chi-square approximation). Once a measure is chosen, the distribution of the combined measure can be approximated from the simulated bivariate distribution, obtaining a combined p-value that accounts for the dependence.

  • 1
    $\begingroup$ Why the cubic root transform? I'm not questioning it, but how did you pick that transform and not, say, log? $\endgroup$ – mugen Mar 3 '17 at 11:55
  • $\begingroup$ @mugen I wanted approximate symmetry because that often makes it a bit easier to assess the dependence/independence. For iid normal $X_i$, the sample variance has a gamma distribution (scaled chi-square), for which the Wilson-HIlferty transformation - in essence the cube root - produces an approximately symmetric (indeed, approximately normal) result. I then used the same transformation on the other example (I didn't expect it to produce symmetry there, but it was still handy for seeing the dependence in that case) $\endgroup$ – Glen_b Mar 3 '17 at 12:00
  • $\begingroup$ @Glen_b Thank you, this is very useful thing to know! I was sure your choice had some theoretical background (not just an arbitrary pro tip). $\endgroup$ – mugen Mar 3 '17 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.