I just came across a correlation factor of 0.70 but p-value 0.054, does this mean that my correlation factor is completely meaningless and I should try to provide more samples?
A p-value above 0.05 doesn't necessarily say 'your correlation is meaningless'.
However, there's more than a 5% chance that you could see a sample correlation at least as far from zero when the population correlation is zero.
Loosely this means you can't confidently distinguish the population correlation your sample was drawn from, from one that is zero (assuming you do mean to set your significance level to 5%)
It depends on what you are trying to do. I frequently estimate models where I literally do not care about the "p" values because I believe my model. The best estimate of the model is the estimate, not the value that the estimate may or may not be significantly different from.
On the other hand if the purpose is the test a binary hypothesis and not to fit a model, then your results still may or not be "meaningless". The following is a non-comprehensive list of scenarios that I've experienced:
- Textbook interpretation: Taking your "p" value as the "true" p value then you can interpret your results as "meaningless" at a 5% level but significant at a 10% level.
- Your "p" value may be inaccurate. It was created using a set of assumptions that may or may not be satisfied in your test. Your actual "p" value may differ depending on whether the assumptions are satisfied.
- Your entire model may be incorrectly specified and any "p" values (whether "significant" or not) are meaningless because your model doesn't actually approximate the data generating process.
On a last note: no data is meaningless. However, I interpret your use of the word "meaningless" to mean "not significant".
The p-value is a measure of the evidence against the null hypothesis provided by the data: the smaller the p-value, the stronger the evidence against the null. Typically, researchers use the following evidence scale:
- p(X) < 0.01 very strong evidence,
- p(X) ∈ (0.01, 0.05) strong evidence,
- p(X) ∈ (0.05, 0.1) weak evidence,
- p(X) > 0.1 little or no evidence.
Using to this "classification" as a benchmark, I would not call your results "meaningless", since there is some evidence against the null. I would try to collect and incorporate more data into the analysis.
You can use a expression like "marginally significant under 0.06 significance level". 0.05 is popular but not absolute.