5
$\begingroup$

I just came across a correlation factor of 0.70 but p-value 0.054, does this mean that my correlation factor is completely meaningless and I should try to provide more samples?

$\endgroup$
8
$\begingroup$

A p-value above 0.05 doesn't necessarily say 'your correlation is meaningless'.

However, there's more than a 5% chance that you could see a sample correlation at least as far from zero when the population correlation is zero.

Loosely this means you can't confidently distinguish the population correlation your sample was drawn from, from one that is zero (assuming you do mean to set your significance level to 5%)

$\endgroup$
3
$\begingroup$

It depends on what you are trying to do. I frequently estimate models where I literally do not care about the "p" values because I believe my model. The best estimate of the model is the estimate, not the value that the estimate may or may not be significantly different from.

On the other hand if the purpose is the test a binary hypothesis and not to fit a model, then your results still may or not be "meaningless". The following is a non-comprehensive list of scenarios that I've experienced:

  1. Textbook interpretation: Taking your "p" value as the "true" p value then you can interpret your results as "meaningless" at a 5% level but significant at a 10% level.
  2. Your "p" value may be inaccurate. It was created using a set of assumptions that may or may not be satisfied in your test. Your actual "p" value may differ depending on whether the assumptions are satisfied.
  3. Your entire model may be incorrectly specified and any "p" values (whether "significant" or not) are meaningless because your model doesn't actually approximate the data generating process.

On a last note: no data is meaningless. However, I interpret your use of the word "meaningless" to mean "not significant".

$\endgroup$
1
$\begingroup$

The p-value is a measure of the evidence against the null hypothesis provided by the data: the smaller the p-value, the stronger the evidence against the null. Typically, researchers use the following evidence scale:

  • p(X) < 0.01 very strong evidence,
  • p(X) ∈ (0.01, 0.05) strong evidence,
  • p(X) ∈ (0.05, 0.1) weak evidence,
  • p(X) > 0.1 little or no evidence.

Using to this "classification" as a benchmark, I would not call your results "meaningless", since there is some evidence against the null. I would try to collect and incorporate more data into the analysis.

$\endgroup$
  • 2
    $\begingroup$ I've seen such use before, but I'm not quite confident if they are in fact statistically correct. This blog discusses this: blog.minitab.com/blog/understanding-statistics/… $\endgroup$ – rph Jun 19 '16 at 11:28
  • $\begingroup$ I agree that this classification is somewhat subjective. But people need to have some classification for decision-making purposes. $\endgroup$ – Kostia Jun 19 '16 at 17:55
  • 2
    $\begingroup$ For decision making purposes, there's decision theory... Also, if you're going to get all set theory and intervals on us then some of yours probably ought to be closed. $\endgroup$ – conjugateprior Jun 19 '16 at 17:57
  • $\begingroup$ I hope you would not argue that p-values are used for accepting or rejecting hypotheses :) Accepting or rejecting a scientific hypothesis is an example of decision making. Decision theory is good in theory :) $\endgroup$ – Kostia Jun 22 '16 at 7:34
1
$\begingroup$

You can use a expression like "marginally significant under 0.06 significance level". 0.05 is popular but not absolute.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.