Getting pearson corelation coefficient greater than 1

Question

I was learning about the metrics which measure the relationship between the variables. I wrote a python code that can generate and calculate covariance and person correlation coefficient.

import numpy as np
from numpy import mean
from numpy import std
from numpy.random import randn
from numpy.random import seed
from matplotlib import pyplot
# seed random number generator
seed(1)
# prepare data
data1 = 20 * randn(1000) + 100
data2 = data1 + (10 * randn(1000) + 50)

When I calculate covariance, I get values like [[385.33, 389.75],[389.75, 500.38]], which looks fine to me. But when I calculate Pearson correlation coefficient using below code,

np.cov(data1,data2) / ((np.std(data1) * np.std(data2)))

I get the values as array([[0.87842078, 0.88850041], [0.88850041, 1.14068681]])

The correlation of X with X should be one, and Y with Y should be one. But I am getting some other values. Is my understanding correct?

Answer 1

Let’s define $\Sigma$ as the covariance matrix.

You did $\dfrac{\Sigma}{\sigma_X\sigma_Y}$. In other words, divide each entry of the matrix by the standard deviations.

This is not correct for the diagonal elements. The diagonal elements should be divided by the variance of the corresponding random variable, so $\sigma_X\sigma_X$ for the top left and $\sigma_Y\sigma_Y$ for the bottom right.

As stated in the comments, numpy.corrcoef does the correlation matrix for you, so you do not have to implement it yourself.