Given a matrix $X$ and the resulting sample correlation matrix $R$, consider the standardized observations:

$$\frac{(x_{jk} - \bar x)} {\sqrt{S_{kk}}} \quad k=1,2,...,p \quad j=1,2,...,n$$

Show that these standardized quantities have sample covariance matrix $R$.

Sample covariance is defined as:

$$\frac{1}{n} \sum\limits_{j=1}^n{(x_{ji} - \bar x_i)(x_{ji} - \bar x_k)}$$

Sample correlation coefficient:

$$r_{ik}=\frac{S_{ik}} {\sqrt{S_{ii}}\sqrt{S_{kk}}} = \frac{\sum\limits_{j=1}^n(x_{ji}-\bar x_i)(x_{jk}-\bar x_k)}{\sqrt{\sum\limits_{j=1}^n(x_{ji}-\bar x_i)^2\sum\limits_{j=1}^n(x_{jk}-\bar x_k)^2}}$$

It is also known that: $$R (correlation) = D^{-1/2}SD^{-1/2}$$ S = variance-covariance matrix

D = sample standard deviation matrix

How do I show that for standardized quantities sample correlation is just the sample covariance?

Attempted answer: For standardized variable, variance is defined as: $$\frac{1}{n}\sum_j\frac{(x_{jk} - \bar x_k)^2} {S_{kk}}=\frac{S_{kk}}{S_{kk}}=1, \quad k=1,2,...,p,$$

thus D = diagonal matrix with variance satisfies: $$ D = I $$ and since $$R (correlation) = D^{-1/2}SD^{-1/2}$$ we can conclude that $$ R=D $$

  • $\begingroup$ By computing $D$? $\endgroup$ – Vincent Guillemot Mar 8 at 10:52
  • $\begingroup$ @VincentGuillemot Sorry I made a typo, should be -1/2 instead of 1/2. $\endgroup$ – lydias Mar 8 at 13:30
  • $\begingroup$ @VincentGuillemot What if we do not know the last condition? How do we prove based only on knowing it's standardized quantities? $\endgroup$ – lydias Mar 8 at 13:32

The variance of a standardized variable is $$\frac{1}{n}\sum_j\frac{(x_{jk} - \bar x_k)^2} {S_{kk}}=\frac{S_{kk}}{S_{kk}}=1, \quad k=1,2,...,p,$$ and so is, therefore, its standard deviation. Hence, $D$, which is a diagonal matrix with the variances (not the standard deviations, that is given by $D^{1/2}$) on the main diagonal, satisfies $D=I$, and therefore, $D^{1/2}=D^{-1/2}=I$, the identity matrix.

Notice we do not need to subtract the sample average, as a demeaned variable has mean zero by construction: $\sum_{j=1}^n(x_j-\bar x)=\sum_{j=1}^nx_j-n\bar x=0$.

  • $\begingroup$ Can I say that since standardized values are centered at zero and expressed in standard deviation units. The sample correlation is equal sample covariance ? Also D^{1/2}RD^{1/2} = S -> R=S? $\endgroup$ – lydias Mar 9 at 5:49
  • $\begingroup$ But how should I express this concept in terms of correlation coefficient formula? $\endgroup$ – lydias Mar 9 at 8:18
  • $\begingroup$ I gave some further hints. $\endgroup$ – Christoph Hanck Mar 9 at 10:13
  • $\begingroup$ @christophhack I was on vacation and just got back on Tuesday. I posted an attempted answer, and I hope it not too simple. Thank you for all the guidance. $\endgroup$ – lydias Mar 22 at 4:11
  • $\begingroup$ @lydias, yes, that properly uses the hints from my answer $\endgroup$ – Christoph Hanck Mar 22 at 9:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.