Interpreting high p value and low correlation value

Question

I am trying to run regression on financial data in R. I am new to regression analysis so I am finding it to difficult to interpret certain scenarios. I have the code as follows:

#regression analysis
fit <- lm(fiveMinReturns~RegressionData, data=maindata)
summary(fit) # show results
#correlation
cor(maindata$fiveMinReturns,maindata$RegressionData,use="everything")

My output is:

Call:
lm(formula = fiveMinReturns ~ RegressionData, data = maindata)

Residuals:
      Min        1Q    Median        3Q       Max 
-0.205790 -0.001144 -0.000062  0.001117  0.156418 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)    6.346e-05  8.785e-06   7.223 5.09e-13 ***
RegressionData 1.597e-07  1.432e-08  11.155  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.004035 on 210912 degrees of freedom
Multiple R-squared:  0.0005896, Adjusted R-squared:  0.0005849 
F-statistic: 124.4 on 1 and 210912 DF,  p-value: < 2.2e-16

cor(maindata$fiveMinReturns,maindata$RegressionData,use="everything")
[1] 0.02428219

p-value is very small that means two variables are tightly coupled, but correlation is small too. My question is how do I evaluate this situation? Can we say that this equation will give correct results almost every time? Which scenario suggests both p-value and correlation both to be really small? What measures should i take to improve the result?

Answer 1

My question is how do I evaluate this situation?

It means that the relationship between your two variables is very small, and only detected because your analyses had massive power (df of 200k+).

Can we say that this equation will give correct results almost every time?

No, again, because the relationship between the two variables is very small, and your data do not explain a lot of the underlying variability in your dependent measure. Specifically, your model has an R^2 of 0.0005849, which means that your independent variable explains .05% (not 5%) of all the variability in the dependent variable.

Which scenario suggests both p-value and correlation both to be really small?

If there is a relationship between the two variables, regardless of how large or small (i.e., H1 is true), the p-value will become smaller as your sample grows larger. So, it is entirely possible that the relationship between your variables is tiny, and yet you've still managed to detect it because your sample is so huge.

What measures should i take to improve the result?

I'm not sure what you mean. The result is fine as it is. You tested a hypothesis and the data do not really support it.

Answer 2

One scenario would be a huge sample size (I'm guessing you have a very big data set), and some small effect that is significant statistically in that it explains some portion of variance (less than 1% in your case), but not particularly useful from a predictive standpoint if your goal is to predict the FiveMinReturns variable.

Answer 3

Significance addresses whether or not the data are similar to the null hypothesis. Specifically, the p-value indicates the probability of observing a correlation as strong as the one you just observed (or stronger) , if the null hypothesis (i.e. 'no correlation') really were true.

With a huge sample size (210912 DF), you have great power to detect even a small divergence from the null), which is what you are observing.

I often find that people confuse the concepts of meaningful (is the correlation big enough to care about) versus significance (is the data consistent with the null hypothesis).

Answer 4

You can see what that means visually by checking this paper in PeerJ. They show an example of a significant relationship with low correlation.

https://peerj.com/articles/589/

Interpreting your results is more than just R2 and p-values.

Your regression coefficient is 0.00000016, meaning that for a one unit change in X, you only get a 0.00000016 unit change in y. This is a very tiny effect size. However, as others have pointed out, you have a giant sample size, so your standard error of that estimate is so low that you're virtually guaranteed to reject the hypothesis that there is no effect. Next, looking at the R2 value, it suggests that your X data have little predictive or explanatory power for you response. So your point estimates of mean responses (basically the regression line) provide very little information about the individual responses. The higher the R2, the closer the scatter of points is to the regression line, the better the regression line approximates the individual response. You need to consider all three of these things separately to get a solid interpretation.