1
$\begingroup$

Most examples in my Time Series Analysis slides and Multivariate Analysis textbook (Applied Multivariate Statistical Analysis, 6th Edition) conduct a hypothesis testing on data's normality, or zero correlation, or independence, or homogeneity of variance, or whatever, and readily assume these properties to be true on failure to reject the hypothesis. Actually, I am explicitly taught that a property holds if it cannot be rejected.

For example, in Example 6.14 (A profile analysis of love and marriage data) from the textbook AMSA, authors "conclude that the hypothesis of parallel profiles for men and women is tenable" when $H_0$ cannot be rejected. In addition, they also said "... we cannot reject the hypothesis that the profiles are coincident. That is, the responses of men and women to the four questions posed appear to be the same." Here is the whole example: P.325, P.326, P.327, P.328, where the quotes are from the bottom of P.327 and the top of P.328 respectively.

This seems ridiculous, as absence of evidence is not evidence of an absence. Even if $p = 1$, it's still circular reasoning, let along cases where $p$ isn't even remotely close to 1. Back when I was learning Mathematical Statistics, the professor would kick their ass if someone accept $H_0$ on failure to reject it, but the standard is clearly loosened a lot in upper division courses.

I just want to know is this practice a de facto procedure in intermediate Statistic courses like Time Series Analysis and Multivariate Analysis? If so, what the rationale?

$\endgroup$
14
  • $\begingroup$ Can you provide more context? Your examples (normality, etc.) suggest these claims have to do with assumptions of inferential tests/procedures. If so, while never strictly true (as you say), a lack of evidence for non-normality of residuals (for example) is taken as evidence that the assumption of normality is approximately true; i.e. the deviation is not so strong that you need to alter your procedures & conclusions. $\endgroup$
    – mkt
    May 8, 2019 at 8:23
  • $\begingroup$ @mkt Hi, I'm not sure what do you mean by "context", but on failure to reject normality, the authors usually go on to apply statistical methods only applicable to data drawn from a (multivariate) normal distribution. Another example is Example 6.14 (A profile analysis of love and marriage data) from the textbook AMSA, where authors "conclude that the hypothesis of parallel profiles for men and women is tenable" when $H_0$ cannot be rejected. $\endgroup$
    – nalzok
    May 8, 2019 at 8:36
  • $\begingroup$ Most of us do not have access to the book you are referring to, so relevant quotations that expand on your claims about what the authors say is what I meant by 'context'. In the one quote you provide, concluding that two groups being similar 'is tenable' based on a failure to reject H0 hardly seems like a radical claim. $\endgroup$
    – mkt
    May 8, 2019 at 8:50
  • 1
    $\begingroup$ @mkt The whole example is way too long to be typed by hand, so I have included it as screenshots in the original post. Hope you don't mind. $\endgroup$
    – nalzok
    May 8, 2019 at 9:06
  • 4
    $\begingroup$ @NickCox I see, so hypothesis testing is more of a means to check/verify the data at hand don't drift too much from the ideal conditions of inference procedures to be employed, but still, nothing can be proved from this check. I will know if my assumptions are absurdly wrong, but there is a chance that the data will pass the check while the assumptions aren't meet, but that's OK since tiny drifts don't invalidate the applicability of inference procedures. $\endgroup$
    – nalzok
    May 8, 2019 at 11:12

1 Answer 1

2
$\begingroup$

Here is a demonstration using four samples of size $n = 15$ from uniform, gamma, beta, and normal populations.

Tests of normality. In all four cases the data 'pass' a Shapiro-Wilk test for normality. That is, we cannot reject the null hypothesis that data are normal. Approximate P-values for the normality test are 0.40, 0.48, 0.09, and 0.36, respectively--all clearly above 0.05.

The PDFs of the first three distributions are all substantially different from normal, but $n = 15$ observations are not enough to detect the departure from normality.

Robustness of the t test: Also, in all four cases we use a one-sample t test to test a null hypothesis about the population mean that is false. For all four samples, the null hypothesis is (correctly) rejected. Even for a sample size as small as $n = 15,$ the test detects that the various hypothetical values of the population mean are not correct.

The t test is said to be 'robust' against departures from normality---as long as there are no extreme outliers. Although the first three samples are non-normal, the t test still gives useful results.

Here are boxplots of the four samples;

enter image description here

set.seed(1234);  n = 15
u = runif(n);  shapiro.test(u)$p.val;  t.test(u, mu=.7)$p.val
[1] 0.3963761   # P-value for normality test
 [1] 0.0145073  # P-value for t test

v = rgamma(n, 5, 1); shapiro.test(v)$p.val; 
t.test(v, mu=2)$p.val
[1] 0.4827358
 [1] 5.112448e-05

w = rbeta(n,3,4); shapiro.test(w)$p.val; 
t.test(w,mu=.7)$p.val
[1] 0.08774671
 [1] 9.815778e-05

z = rnorm(n); shapiro.test(z)$p.val; 
t.test(z,mu=.7)$p.val
[1] 0.3592276
 [1] 0.003356458

Note: The seed for the simulation is given so that you can reproduce exactly the same simulated data used here. Different seeds might give substantially different results.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.