5
$\begingroup$

I ran an OLS regression, and the p-value of one variable is 0.065, which is more than the threshold to consider statistically significant. However, it's not too bad either, as my data has only 300+ data points. Therefore I think the explanation needs to be taken with a bit of salt.

How would you explain such a finding to non-stats people?

$\endgroup$
2
  • 2
    $\begingroup$ Since you consider 0.065 not to be statistically significant, then above all else you need to explain that you performed a test that fails to reject the null hypothesis. Every thing else is secondary to that salient fact. (I am very concerned that many of the suggestions made in the answers could be read as ways to hide or obscure this main point. Such actions potentially run afoul of professional ethical guidelines.) $\endgroup$
    – whuber
    Jul 7, 2014 at 16:02
  • $\begingroup$ I know we live in the age of big data, but I am still surprised that you consider 300+ data points to be little for an OLS regression. Regarding your question when you say "one of the variables" it sounds like you have several; did you check for correlations and variance inflations? It might not make sense to interpret the result for that one coefficient in isolation. $\endgroup$
    – Erik
    Jul 9, 2014 at 9:27

6 Answers 6

4
$\begingroup$
  1. Use "words", do not talk to non-technical people about p-values. They won't understand.
  2. Use your domain knowledge. It might be on the borderline of the 5% significance level, but your domain knowledge might tell you that it is an important factor.
  3. If your domain knowledge tells you that the coefficient must be positive (or must be negative), then you can do a one-sided test.
  4. It is still significant at the 10% level anyway.

Edit: To address @whuber's comment: My answer was intended to focus on domain knowledge, not not obscuring the fact that a statistical test was failed. In short, see @whuber's comment.

$\endgroup$
1
3
$\begingroup$

Tell the story you found out, not to explain the maths behind your findings. Non statistical-people wants to know about the stories behind the data. Usually, they use your analysis to take decision in a fast way.

$\endgroup$
3
$\begingroup$

Drop the hypothesis test and the p value altogether. I'm not confident that 95% of "stats people" really understand p values; with "non-stats people", I'd bet it's less than half. Don't go there unless you must.

In your example, it sounds like you're trying to reject the null that the slope parameter for the predictor in question is zero. Is that really what you want to do? Might you prefer to estimate what the slope parameter actually is, and express your degree of confidence in that estimate? My guess is that most non-stats people would prefer the latter. It doesn't just take statistical education to understand a p value; it also takes a fair amount of epistemology to appreciate the value of falsification, and why a big scientific study could somehow not produce good enough results to conclude anything trustworthy enough to deserve printing. Conversely, positively framed information appeals to basic intuition.

Consider presenting your results as effect size estimates with confidence intervals. This won't entail falsificationism, will focus on the information gained from your study, and will communicate what needs to be said about the population from whence you sampled. The slope coefficient itself can be interpreted in simple terms of size (e.g., a standardized $\beta=.15$ is probably a fairly weak relationship, but if you have 3× larger standard error, maybe your $\beta=.45$; that's a fairly strong relationship in many contexts), or in literal terms of the regression equation (e.g., for one unit increase in your predictor, your model estimates that the outcome increases by b). Either way, it's much more intuitive than the correct understanding of a p value. It's somewhat different information, but again, it's probably the information a non-academic audience would want first. They might even have a healthily skeptical attitude toward the results of your study with regard to conclusions about the population based on intuition alone, so I don't think you cause any harm necessarily and inherently by focusing attention on your study's estimates instead of on its implications for the population.

For academic, statistics, and all other kinds of audiences, confidence intervals should suffice as the vehicle of information regarding the population. In your case, while presenting your effect size estimate and suggesting its implications, it may be worthwhile to draw attention to the marginally >5% chance that the real relationship between predictor and outcome could actually operate very (probably even negligibly) weakly in the opposite direction from what your study indicates in the overall population...or it may not. Non-academic audiences may be just as content with a 90% confidence interval that leaves only a 10% doubt that your study has enumerated the correct range of possibilities for the strength of the relationship in question, and if you modeled your data correctly, you can be 90% confident that there is a relationship in the direction your model indicates. Furthermore, a confidence interval draws attention to the possibility that the relationship is larger than your model estimates. IMO, focusing on null hypotheses tends to draw attention toward the more dismissive half of the confidence interval, but that is only half of the story.

$\endgroup$
2
$\begingroup$

Keep it simple.

I believe this correlation to be valid, but with the data I have at the moment, it may or may not be. I need at least twice (or 3X or 10X, etc.) the data to be sure, one way or the other.

If that doesn't get a satisfactory response, consider an analogy.

We all know that home prices are related to square footage, right? If I gathered a small sampling of home sales - say 5 in Beverly Hills, 5 in rural Montana, and 1 in New York City - we probably wouldn't be able to prove it statistically. That's where I'm at now. My instincts tell me this relationship is valid. My data is telling me "maybe, maybe not". It has crossed the threshold into maybe not territory, but just barely.

Regardless, I would stay away from discussing p-values all together. The topic will make a classroom full of students who want to do statistical work for a living fall asleep.

$\endgroup$
2
$\begingroup$

If you think 300 points is limited, than that's the most important argumentation. Hypothesis tests are designed towards keeping the null, unless there is strong evidence against it.

Absence of evidence is not evidence of absence.

I guess also non-stats people should get this.

$\endgroup$
0
$\begingroup$

I think even non-stats people can understand a distribution curve - most people have seen that in high school and/or college. So you could also show it visually in the sense that people will understand rejection regions vis-a-vis your null hypothesis, etc. You could literally shade those in - "here's where we make this decision and here's where our test came up. See how close that is?". I think that would get your desired effect across.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.