0
$\begingroup$

I am currently running a structural equation model, where I have made some directional hypotheses before hand of how the variables will be associated with each other. Specifically, I hypothesized that being female (gender) will have a negative effect on mathematics self-concept, self-efficacy and interest (modeled as latent variables). I did my analyses using MPlus, which only returns two-tailed p-values. Model fit indices and everything were great, I have a question about the specific relationships between variables. My results were as follows:

Estimate S.E. Est./S.E. 2 Tail P-Value
SELF CON ON GENDER -0.171 0.069 -2.161 0.031
SELF EFF ON GENDER -0.351 0.065 -5.093 0.000
INTEREST ON GENDER 0.175 0.067 2.472 0.013

As you can see, I did find significant negative effects for self-concept and self-efficacy. I understand that since I had a directional hypothesis, I should convert this two tailed p-value into a one-tail value by dividing it in half (therefore, for self-concept, the p-value would be around 0.015). However, I am struggling with the relationship between gender and interest. As you can see, there was also a significant effect from the two-tailed test, however, the effect was in the opposite direction of what I predicted. Originally, I thought this just meant I had found an interesting result. However, after reading some resources online [UCLA: What is a two-tailed test?](e.g., https://stats.oarc.ucla.edu/other/mult-pkg/faq/pvalue-htm/) it seems that if you had a directional hypothesis, but find an effect in the opposite direction, you should compute the one-tail p-value by doing 1 - (pvalue/2) BECAUSE the effect was not in the direction you expected. In this case, there would be NO significant effects of gender on interest. However, I am not sure this is correct.

Would someone be able to confirm the correct way to proceed? Report the effect of gender on interest as significant or not? Or provide any resources that might help me out here?

$\endgroup$

1 Answer 1

1
$\begingroup$

if you had a directional hypothesis, but find an effect in the opposite direction, you should compute the one-tail p-value by doing 1 - (pvalue/2) BECAUSE the effect was not in the direction you expected. In this case, there would be NO significant effects of gender on interest. However, I am not sure this is correct.

Yes, it is correct. Your data are consistent with your $H_0$ that the effect is nonnegative because the estimated effect is in fact nonnegative. This result is easier to understand if your test your $H_0$ by comparing your obtained z statistic to a critical value, instead of comparing a p value to an $\alpha$ level. But they are equivalent, so here is some detail that might help.

If you test $H_0: \beta=0$ against $H_A: \beta \ne 0$

This is not your situation, but it is the default situation assumed by the software that provides 2-tailed p values, so it helps to start here. Note that I only discuss using a test statistic with a(n assumed) symmetric distribution (e.g., t or normal), as an F or $\chi^2$ distribution is implicitly appropriate only for nondirectional hypotheses.

A p value is the probability that a z statistic exceeds your obtained one, if the $H_0$ is true. Using your $z=2.472$, you can replicate the software's calculated probability (of one at least that large if $H_0$ is true) using the pnorm() function:

pnorm(2.472, lower.tail = FALSE) + pnorm(-2.472, lower.tail = TRUE)

By default, that is for a nondirectional hypothesis (i.e., $H_A: \beta \ne 0), so the first term calculates the upper-tail probability of a larger test statistic, and the second term calculates the lower-tail probability of a smaller (i.e., more negative) test statistic. Both probabilities are the same, so you could also just double the upper-tail p value of the absolute value of a test statistic:

2*pnorm(abs(2.472), lower.tail = FALSE)

Equivalently, the critical values would be obtained by splitting the Type I error rate equally across both tails, because either direction would be considered equally interesting/surprising if $H_0$ is true.

alpha <- .05

# reject H0 when z < -1.959964
qnorm(    alpha/2 , lower.tail = TRUE ) 

# reject H0 when z > 1.959964
qnorm(1 - alpha/2 , lower.tail = FALSE)

If you test $H_0: \beta=0$ against $H_A: \beta > 0$

This is also not your situation, but your result would not contradict this $H_A$, so let's talk about this case first. We no longer split up our rejection region across tails, and the critical value would only be the positive one:

# reject H0 when z > 1.644854
qnorm(alpha, lower.tail = FALSE)

Because the effect is in the expected direction, the p is also only in the upper tail (i.e., in the same direction as the $H_A$):

pnorm(2.472, lower.tail = FALSE)

If you test $H_0: \beta=0$ against $H_A: \beta < 0$

The critical value for this directional $H_A$ is negative, so you need to obtain a z statistic more negative (smaller) than this one:

# reject H0 when z < -1.644854
qnorm(alpha, lower.tail = TRUE)

Your obtained z does not allow you to reject $H_0$, which I'm sure you understand. But conducting an equivalent test using a corresponding p value is the tricky part that inspired your question. We should be calculating a p value the represents the cumulative area under the normal-curve in the same direction as $H_A$ (like the situations above). That would mean calculating the area below your obtained z statistic:

pnorm(2.472, lower.tail = TRUE)

This is equivalent to how the UCLA article you linked to describes how to obtain the correct p value from the one reported by your software. For more precision, let's grab the 2-tailed p value from pnorm that I showed above:

(reported.p.value <- 2*pnorm(abs(2.472), lower.tail = FALSE))
1 - (reported.p.value / 2)
# same as:
pnorm(2.472, lower.tail = TRUE)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.