2
$\begingroup$

I computed rolling correlations for each 25 quarter period in my 200 quarter x and y datasets. Each p-value is greater than 5%, and is therefore “insignificant.”

All I want to know is if my rolling correlations have ANY value. I’m really interested in the degree to which x and y are correlated in each 25 quarter period. I do not intend for the 25 quarter correlation to be representative of the correlation between all 200 x values and all 200 y values. Let’s say for one of my 25 quarter periods the correlation is 100%, but p-value is 95%. Is it not still true that in that particular 25 quarter period, x and y were in fact 100% correlated???

$\endgroup$
9
  • 1
    $\begingroup$ ANY value? Well, they're telling you that there isn't statistically significant correlation across the window size you've chosen. Could readily be due to high variance and/or non-linearity. $\endgroup$
    – Galen
    Jun 29, 2021 at 2:43
  • $\begingroup$ The more I read the more complicated this is… corrected vs uncorrected p values due to an issue with applying standard t tests on rolling correlations. I don’t think you need to have them be significant to tell you something about a given window. $\endgroup$ Jun 29, 2021 at 4:22
  • 1
    $\begingroup$ The p-value gives the likelihood of a data-generating process giving a test score as-or-more extreme than observed. Often this is taken to be identical and independent normal distributions, but in time series you could consider a variety of stochastic processes including Gaussian processes. $\endgroup$
    – Galen
    Jun 29, 2021 at 4:49
  • $\begingroup$ The adjust p-value is accounting for the fact that the family-wise error rate increases with the number of tests you perform. $\endgroup$
    – Galen
    Jun 29, 2021 at 4:50
  • $\begingroup$ Understood. What do you make of this? search.r-project.org/CRAN/refmans/dLagM/html/sdPercentiles.html $\endgroup$ Jun 29, 2021 at 4:51

1 Answer 1

4
$\begingroup$

Without actually being able to see how you are doing what you are doing, I cannot give a definitive answer. However, for purposes of discussion, we will assume your code is correct.

I see a lot of separate issues in your question.

First, we do not know if Pearson's correlation coefficient is an appropriate metric. If you were looking at accounting data, such as accounts receivables versus inventories, on a quarterly basis, then it would be appropriate. On the other hand, for many types of financial returns, it would not be. So you should dig into the assumptions of Pearson's coefficient and see if they are satisfied. It is not enough to assume that they are satisfied, they actually must be satisfied.

You are using rolling periods, why? If it is because everybody else does it, then that is a bad reason. If a correlation is a stable property of the population, then it is stationary. If you are saying that it is not stationary, why would it happen to correspond to your particular window? Whether you intended to or not, you are asserting that some market changes its mind exactly every three months. If that is not at least an approximately true statement, then the correlations will be bogus.

For example, let us imagine that components of the US stock markets break about every 73 days, on average, but sometimes only last 20 days and have lasted as long as 700 days. That will totally blow up your fixed quarterly window.

Assuming that your null hypothesis is that there are no correlations in the entire set, and if that null is not rejected, then that implies that your correlations could have been generated by chance effects alone. That would make your statement, "if it were 100% correlated" to be like the case where many people are tossing coins and one tossed a coin as heads ten times in a row and asking "wasn't that one coin tosser the best?"

If you see two people walking down the street together, it could imply that they are best friends going to a common destination. It could also imply that they are strangers that intersected and at the moment are going in the same general direction. At some point, one may go to a coffee shop, while the other catches a bus to go home. With very many interactions, that is bound to happen from time to time. The correlation of the two paths only is meaningful in the common destination case, not the happenstance case.

Additionally, p-values are not unique. There are very well known examples where two researchers observing the same data will get two different p-values as p-values are model dependent. The intentions of the researcher can determine a p-value. In your case, because you are doing multiple comparisons, there will be a correction factor. The multiple comparison rules are not unique, so two researchers observing the same data can get differing p-values if they employ different rules for multiple comparisons.

EDIT Your tool you mentioned in the comments might be inappropriate.

It assumes that $x_{t+1}=x_t+\epsilon_{t+1}$, but for many financial series such as stocks, $x_{t+1}=Rx_t+\epsilon_{t+1},R>1,$ such as stock prices. If the return were less than or equal to one, then equity investors would expect systematic losses.

Your tool is assuming that your data does not change, ever. It is a white noise tool. Do you believe your data is made up only of white noise?

As a side note, there is a proof in 1958 that the equation $x_{t+1}=Rx_t+\epsilon_{t+1},R>1,$ has no non-Bayesian solution under quadratic loss, though you could use Theil's regression or quantile regression through the median.

$\endgroup$
11
  • 1
    $\begingroup$ (+1) If it is because everybody else does it, then that is a bad reason.. $\endgroup$
    – Galen
    Jun 29, 2021 at 4:54
  • 1
    $\begingroup$ @user3138766 Without accounting for the noise? Why? One of the main purposes of the hypothesis tests is to draw inferences in the presence presence of noise. $\endgroup$
    – Galen
    Jun 29, 2021 at 5:09
  • 1
    $\begingroup$ A correlation is an estimator of a population parameter. It is unrelated to the actual realizations that happened in the data. If two people are walking down the street side by side, but don't know each other, then it is happenstance. If they know each other, then that can be thought of as a population parameter. When you are measuring correlation, you are estimating a fixed effect in the system. That fixed effect may be a function of something else and so may be changing. $\endgroup$ Jun 29, 2021 at 5:12
  • 1
    $\begingroup$ Imagine two firms that happen to both grow Fuji apples. If that is their only commonality, then that is likely their only source of correlation for their earnings. If one of the groves burns down, then the relationship will change. If one has a systematic plan to add new orchards, and the other does not, then the relationship will still exist, but depend on the number of groves. $\endgroup$ Jun 29, 2021 at 5:15
  • 1
    $\begingroup$ No, it might be 0. A statistic is a measurement that is a function of the data. As the data has noise in it, then the result can entirely be noise with no signal whatsoever. There is no requirement that $\hat{r}=.25$ that implies that was the actual relationship. $\endgroup$ Jun 29, 2021 at 5:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.