My knowledge of stats is pretty basic. Hopefully this has a simple answer.

I have a matrix of time series data for which I've computed covariance values, and normalized those to get correlation values from -1 to 1.

I want to perform a significance test for each of these correlations.

To give a concrete example, say we have heart rate and weight measurements for a patient on each of 5 doctor's office visits over the past year.

These may appear to be correlated based on this small $n$, but I want a significance test that accounts for sample size.

$p$-value is what I'm used to, but there may be something more appropriate.

To complicate things, I am performing linear interpolation (optionally) on missing values

I want to do this calculation manually, not using R or Excel or anything like that. Any resources, formulae or pseudocode you can provide will be appreciated.

Update: I did find this handy table that shows significance cutoffs for n, but I would prefer to compute this on the fly instead of using a lookup table


Example data might look like:

time | heart rate | weight 
1    | 75         | 150
2    | 76         | 153
3    | 77         | 154
4    | 78         | 158
5    | 79         | 160
  • 1
    $\begingroup$ What is the goal of your analysis? Can you tell us what you are trying to learn from you research? This well help us to recommend the best approach for your analysis. Are you really interested only in correlations or are you interested in learning the effect of weight on heart rate or some other "dependent" variable? $\endgroup$ – StatsStudent Dec 22 '18 at 7:33
  • $\begingroup$ It's for biomedical software that allows users to record arbitrary parameters in a log, including vital signs like weight, blood pressure etc but not limited to these. It's arbitrary and entirely up to the user. I'm automatically generating a matrix with correlation coefficients (Pearson's) across all parameters, whatever they might be. The goal of adding the significance test is to let the user set a p value cutoff and indicate whether their correlation is significant based on that cutoff. $\endgroup$ – 111 Dec 22 '18 at 7:53
  • 1
    $\begingroup$ Great, @glyph. Now, can you provide an example of what your raw data look like and your calculated correlation matrix look like? It's not clear to me if you are interested in calculating the correlation between weight and blood pressure for each visit, or if you are interested in calculating the correlation of weight over time (e.g. between office visits) and separately for blood pressure over time. $\endgroup$ – StatsStudent Dec 22 '18 at 16:29
  • $\begingroup$ I've already calculated the normalized covariance (Pearson's correlation coefficient) for the bivariate time series data, not for individual data points. what I'm interested in is the p value for that coefficient. (added example data to question) $\endgroup$ – 111 Dec 22 '18 at 22:43
  • 1
    $\begingroup$ I provided an answer below. What programming language are you using? If you can tell me I can probably update my answer with some psuedocode and an example. $\endgroup$ – StatsStudent Dec 23 '18 at 8:30

If you can assume that that two variables you wish to calculate the correlation on stem from a bivariate normal distribution, then you can use the Pearson product moment correlation coefficient and perform a test of whether or not the correlations are statistically different from zero (i.e. no correlation).

To carry out this test, you must first calculate the pearson correlation coefficient, $r_{12}$ between variables 1 and 2. It sounds as though you have this calculated already, but I include it here for completeness:


where $n$ is the number of observations in your analysis (this should be five in your case according to the description, but you've only included 4 observations in your table - I'm guessing you accidentally left out the 5th observation) and $\bar{y}_1$ and $\bar{y}_2$ are the means of variables 1 and 2 respectively.

Then, you'll test:

$H_0: \rho_{12}=0$ versus $H_1: \rho_{12}\ne0$

where $\rho_{12}$ is the population correlation that you are trying to estimate.

The test statistic is given by:


If the null hypothesis is true, then $t^*$ follows a student $t$ distribution with $n-2$ degrees of freedom. The $t$-value can then be looked up in any standard $t$-distribution table, and if you always have five observations, you could simply hard code the value for the critical value for the $t$-distribution with $5-2=3$ degrees of freedom which would allow you to determine whether or not you reject the null hypothesis from within your program.

If you wanted to calculate the $p$-value for your test, you'd simply need to pass the value of your test statistic along with the degrees of freedom into any program that is capable of calculating the Cumulative Distribution Function (CDF) of the student $t$ distribution and have it return the corresponding probability. Nearly every computing language has the CDF probabilities built in, so you should use the pre-built functions available to you in their statistics libraries. But if you are really interested in "rolling your" own (I can't encourage you enough to NOT do this though), then you can find all sort of psuedocode on the internet for how to do this, but my favorite resource is the classical Numerical Recipes 3rd Edition: The Art of Scientific Computing (see pages 324 and 325). Many of the standard statistical libraries included in software today use the algorithms presented in this classic text.

A word of caution (updated)

  • This test is not very robust if your data are bivariate normal. In theory, the test does not require bivariate normality (only finite variances and a finite covariance), but that being said, the test seems to perform poorly without bivariate normality in small samples. If bivariate normality is not an appropriate assumption, you should consider using a nonparametric statistic like the Spearman Rank Correlation Coefficient and its corresponding test. See this work for more details.
  • Recall that the correlation coefficient measures the strength of a linear association. So, it's quite possible your data are very strongly related to one another, but just not linearly. So for example, data that could be modeled by a sinusoidal function over time ($x_1=heart.rate$ and $x_2=time$for example), may show a correlation of zero, but be strongly related to each other. This is because heart rate might follow a sinusoidal function of time. Thus, the linear association between heart rate and time during the day (increasing heart rate) may cancel out the effective of the linear association between heart rate during the night (or during sleep with decreasing heart rate).
  • $\begingroup$ I'm actually not sure that assuming some of these biometric data will have normal distribution. They could but won't always. For example if a person's heart rate is generally around 70bpm but occasionally spikes to 120, and never drops below 68, that does not sound like a normal distribution. Can skewed data still be considered normal distribution? I don't know. $\endgroup$ – 111 Dec 25 '18 at 20:08
  • 1
    $\begingroup$ @glyph, I've updated my "A word of caution," so please give it a read. Technically, your sample doesn't have to be bivariate normal, but the test performs better if it is. See the referenced work. This CV post might also be of interest to you: stats.stackexchange.com/questions/3730/…. What you might consider doing is having your program report both the Pearson Test as well as the Spearman non-parametric test. Then you can let the user decide which one is more appropriate. If both agree, that's a strong indicator of (continued) $\endgroup$ – StatsStudent Dec 26 '18 at 18:30
  • 1
    $\begingroup$ of correlation. If both tests disagree, caution should be exercised when interpreting the results. That way, you let the report reader decide for him or herself how appropriate the assumptions might be since it seems the reports can be generated with a large number of different variables used for comparisons, some of which might meet certain assumptions, and others which might not. With this approach, you could also through in Kendall's tau test too if you wanted. $\endgroup$ – StatsStudent Dec 26 '18 at 18:33
  • 1
    $\begingroup$ Thanks for the suggestions. I've added the t-value and am using a t-table to look up the p-value. I agree the best option is to let the user select which method they want. Definitely planning to implement that in the future. $\endgroup$ – 111 Dec 26 '18 at 18:40
  • 1
    $\begingroup$ Also, if you are using PHP as you stated in your other comments, you can use this php function: php.net/manual/en/function.stats-cdf-t.php. $\endgroup$ – StatsStudent Dec 26 '18 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.