1
$\begingroup$

I got this problem while computing the pearson correlation of two datasets where one set consists of the same value. For example this pandas DataFrame:

tdf = pd.DataFrame([[1, 2], [1, 2], [1, 2]], columns=["a", "b"])
tdf.corr()

As a results from pandas correlation function I just geht 'NaN' values. I guess it is because the nominator and denominator from pearson formula are zero. But shouldn't the correlation value should be '1'? Do I missunderstand something?

$\endgroup$
  • 1
    $\begingroup$ Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. You already gave the answer, the standard deviation is zero. Why should there be a correlation of 1? Try instead $(1,2,3)$ for both variables. $\endgroup$ – T.E.G. Jan 29 at 13:51
  • 1
    $\begingroup$ Think of this graphically or geometrically. You have three points that will plot at the same position. The algebra says that they all follow $b = 2a$ (in your notation) but the geometry does not define a unique straight-line summary. (I recommend thinking of these as two variables, not two datasets.) $\endgroup$ – Nick Cox Jan 29 at 18:17
1
$\begingroup$

Please see the formula below:

enter image description here

The standard deviations in the denominator are each zero, so you end up dividing by 0, which gives you the NaN value for Corr(A,B).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.