If the Pearson correlation coefficient is small/weak, can it have criterion validity?

Question

The r value I have obtained is -0.190 and p = 0.410. As scores on stress increase, scores on eating habits decrease. I know from my textbook that criterion validity is about whether your measure is correlated with a relevant outcome. In other words, criterion validity evaluates whether the measures under consideration is related to a concrete outcome, such as a behavior, that it should be related to, according to the theory being tested.

One would expect that if there is more stress, eating habits would turn downhill. If the Pearson correlation coefficient is small/weak, can it have criterion validity?

Answer 1

One might consider this from several viewpoints.

1. (This is a knee-jerk answer, the sort that many of us would give if not thinking through the situation very deeply.) A correlation of -.19 is far too weak to provide much evidence of criterion validity either for stress scores or eating habits scores.

2. It's necessary to view a scatterplot of the two variables, because the low linear association might mask a stronger nonlinear relationship. This would be particularly relevant if theory suggested that such a nonlinear relationship existed, for instance if, beyond a certain level, greater stress were expected to affect eating habits much more strongly than was true at lower stress levels.

3. The finding does not support the criterion validity being assessed, but this research really hasn't given it much of a chance. Statistical power in this situation is low; the confidence interval around the point estimate of -.19 is very wide, probably about [-.6, .2]. (From your results I'm guessing N ~ 22 and the standard error of r ~ .2.) So results from this small sample leave plenty of room for a population correlation of -.5 to -.6. Would such a result, if obtained via another sample, be sufficient given your knowledge of the two variables? After all, one would hardly expect stress to explain everything about eating habits.