2
$\begingroup$

I am trying to run regression on financial data in R. I am new to regression analysis so I am finding it to difficult to interpret certain scenarios. I have the code as follows:

#regression analysis
fit <- lm(fiveMinReturns~RegressionData, data=maindata)
summary(fit) # show results
#correlation
cor(maindata$fiveMinReturns,maindata$RegressionData,use="everything")

My output is:

Call:
lm(formula = fiveMinReturns ~ RegressionData, data = maindata)

Residuals:
      Min        1Q    Median        3Q       Max 
-0.205790 -0.001144 -0.000062  0.001117  0.156418 

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)    6.346e-05  8.785e-06   7.223 5.09e-13 ***
RegressionData 1.597e-07  1.432e-08  11.155  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.004035 on 210912 degrees of freedom
Multiple R-squared:  0.0005896, Adjusted R-squared:  0.0005849 
F-statistic: 124.4 on 1 and 210912 DF,  p-value: < 2.2e-16

cor(maindata$fiveMinReturns,maindata$RegressionData,use="everything")
[1] 0.02428219

p-value is very small that means two variables are tightly coupled, but correlation is small too. My question is how do I evaluate this situation? Can we say that this equation will give correct results almost every time? Which scenario suggests both p-value and correlation both to be really small? What measures should i take to improve the result?

$\endgroup$
  • $\begingroup$ Sorry for that. $\endgroup$ – Rishi Bhatt Jan 14 '15 at 23:41
  • 2
    $\begingroup$ The output cannot be interpreted in the usual way, because five-minute returns on any financial instrument are likely to exhibit strong serial correlations across long lags. That means a key assumption behind the p-value calculation--that the residuals are independent--is seriously violated. A bit of advice: when you're new to any topic in data analysis, don't try to learn it by applying it to datasets with hundreds of thousands of records. Learn on small datasets (up to a few hundred records at most) that are rapidly processed, easily visualized, and fully understood. $\endgroup$ – whuber Jan 15 '15 at 16:15
  • $\begingroup$ Thank you. I will try to run regression on smaller data. $\endgroup$ – Rishi Bhatt Jan 15 '15 at 21:45
4
$\begingroup$

My question is how do I evaluate this situation?

It means that the relationship between your two variables is very small, and only detected because your analyses had massive power (df of 200k+).

Can we say that this equation will give correct results almost every time?

No, again, because the relationship between the two variables is very small, and your data do not explain a lot of the underlying variability in your dependent measure. Specifically, your model has an R^2 of 0.0005849, which means that your independent variable explains .05% (not 5%) of all the variability in the dependent variable.

Which scenario suggests both p-value and correlation both to be really small?

If there is a relationship between the two variables, regardless of how large or small (i.e., H1 is true), the p-value will become smaller as your sample grows larger. So, it is entirely possible that the relationship between your variables is tiny, and yet you've still managed to detect it because your sample is so huge.

What measures should i take to improve the result?

I'm not sure what you mean. The result is fine as it is. You tested a hypothesis and the data do not really support it.

$\endgroup$
1
$\begingroup$

One scenario would be a huge sample size (I'm guessing you have a very big data set), and some small effect that is significant statistically in that it explains some portion of variance (less than 1% in your case), but not particularly useful from a predictive standpoint if your goal is to predict the FiveMinReturns variable.

$\endgroup$
  • $\begingroup$ Thank you hdizzle. Do you mean to say that small p-value does not make any sense? $\endgroup$ – Rishi Bhatt Jan 14 '15 at 23:40
1
$\begingroup$

Significance addresses whether or not the data are similar to the null hypothesis. Specifically, the p-value indicates the probability of observing a correlation as strong as the one you just observed (or stronger) , if the null hypothesis (i.e. 'no correlation') really were true.

With a huge sample size (210912 DF), you have great power to detect even a small divergence from the null), which is what you are observing.

I often find that people confuse the concepts of meaningful (is the correlation big enough to care about) versus significance (is the data consistent with the null hypothesis).

$\endgroup$
1
$\begingroup$

You can see what that means visually by checking this paper in PeerJ. They show an example of a significant relationship with low correlation.

https://peerj.com/articles/589/

Interpreting your results is more than just R2 and p-values.

Your regression coefficient is 0.00000016, meaning that for a one unit change in X, you only get a 0.00000016 unit change in y. This is a very tiny effect size. However, as others have pointed out, you have a giant sample size, so your standard error of that estimate is so low that you're virtually guaranteed to reject the hypothesis that there is no effect. Next, looking at the R2 value, it suggests that your X data have little predictive or explanatory power for you response. So your point estimates of mean responses (basically the regression line) provide very little information about the individual responses. The higher the R2, the closer the scatter of points is to the regression line, the better the regression line approximates the individual response. You need to consider all three of these things separately to get a solid interpretation.

$\endgroup$
  • 1
    $\begingroup$ Whether $0.00000016$ is "tiny" depends on the units of measurement of all the variables. For instance, if RegressionData is a time variable measured in seconds (which is common in computing systems), converting it to years would change the coefficient to $5.05$. Is that "tiny," too? Although normally--when suitable units are used--a value like $0.00000016$ would be considered small, in the present circumstance you can't assume the units were well chosen. You simply don't have the information needed to decide whether this coefficient is small, large, or anything in between. $\endgroup$ – whuber Jan 15 '15 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.