I was learning about the metrics which measure the relationship between the variables. I wrote a python code that can generate and calculate covariance and person correlation coefficient.

import numpy as np
from numpy import mean
from numpy import std
from numpy.random import randn
from numpy.random import seed
from matplotlib import pyplot
# seed random number generator
# prepare data
data1 = 20 * randn(1000) + 100
data2 = data1 + (10 * randn(1000) + 50)

When I calculate covariance, I get values like [[385.33, 389.75],[389.75, 500.38]], which looks fine to me. But when I calculate Pearson correlation coefficient using below code,

np.cov(data1,data2) / ((np.std(data1) * np.std(data2)))

I get the values as array([[0.87842078, 0.88850041], [0.88850041, 1.14068681]])

The correlation of X with X should be one, and Y with Y should be one. But I am getting some other values. Is my understanding correct?

  • 2
    $\begingroup$ Question is more suited to stackoverflow. Please use np.corrcoef. $\endgroup$
    – msuzen
    Nov 23 '21 at 17:04

Let’s define $\Sigma$ as the covariance matrix.

You did $\dfrac{\Sigma}{\sigma_X\sigma_Y}$. In other words, divide each entry of the matrix by the standard deviations.

This is not correct for the diagonal elements. The diagonal elements should be divided by the variance of the corresponding random variable, so $\sigma_X\sigma_X$ for the top left and $\sigma_Y\sigma_Y$ for the bottom right.

As stated in the comments, numpy.corrcoef does the correlation matrix for you, so you do not have to implement it yourself.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.