# Why does the correlation function in R, cor() return a matrix with fewer rows that you started with?

I have a matrix $G$ in which I would like to compute the correlation matrix of. The matrix in R looks like:

> G
[,1] [,2] [,3] [,4] [,5] [,6]
[1,]   -1    0   -1   -1    0   -1
[2,]   -1    0   -1    1    1    0
[3,]   -1    0    1   -1    0    1
[4,]   -1    0    1    1   -1    0
[5,]    1   -1    0   -1    0    1
[6,]    1   -1    0    1   -1    0
[7,]    1    1    0   -1    0   -1
[8,]    1    1    0    1    1    0


Now, I would like to find the correlation matrix. However, when I use the function cor(), I get the following:

> cor(G)
[,1] [,2] [,3] [,4] [,5] [,6]
[1,]    1  0.0  0.0    0  0.0  0.0
[2,]    0  1.0  0.0    0  0.5 -0.5
[3,]    0  0.0  1.0    0 -0.5  0.5
[4,]    0  0.0  0.0    1  0.0  0.0
[5,]    0  0.5 -0.5    0  1.0  0.0
[6,]    0 -0.5  0.5    0  0.0  1.0


Why were two rows omitted here? Am I missing something?

• To whom it may concern, it seems to me this is really a statistical understanding issue, even though it is couched in terms of R. IMO, this should be considered on topic here. Feb 1 '16 at 18:02

Correlation is calculated between columns and not between rows.

The output should be read as, correlation between column-i and column j.

Since you have 6 columns, you get a 6x6 correlation matrix. All 8 rows have been considered while calculating these correlations.

In the $G^TG$ operation, $G^T$ is an $6 \times 8$ matrix, and $G$ an $8 \times 6$. Hence, the matrix multiplication will yield a $6 \times 6$ matrix.

Addressing the comments and the underlying issue, let's pretend that we have a matrix corresponding to returns of different stocks (in the columns) versus 5 consecutive years (in the rows) - completely fictitious stocks and years. Let's call the matrix, $A$:

$$A = \begin{bmatrix} & \color{red}{\text{yah(y)}} & \color{blue}{\text{goog(g)}} & \color{green}{\text{ms(m)}} \\ \text{Yr.1} & 1 &8 & 1\\ \text{Yr.2} & -4 &9 & 3 \\ \text{Yr.3} & 5 & 10 & 4 \\ \text{Yr.4} & 7 & 3 & 5\\ \text{Yr.5} & 8 & 7& 6 \end{bmatrix}$$

We want to calculate the correlations between the different vectors of returns, one for each company, "packaged" in the matrix $A$.

The variance-covariance matrix of the portfolio assuming equal holdings will be:

$\Large \sigma(A) = \frac{G^TG}{n-1}$ with $G$ being the mean-centered observations and $n-1$ corresponding to the number of observations minus $1$.

The mean-centered (or demeaned) matrix $G$ is:

$$\begin{bmatrix} & \color{red}{\text{y}} & \color{blue}{\text{g}} & \color{green}{\text{m}} \\ \text{Yr.1} & -2.4 &0.6 & -2.8\\ \text{Yr.2} & -7.4 &1.6 & -0.8 \\ \text{Yr.3} & 1.6 & 2.6 & 0.2 \\ \text{Yr.4} & 3.6 & -4.4 & 1.2\\ \text{Yr.5} & 4.6 & -0.4& 2.2 \end{bmatrix}$$

And the variance-covariance matrix:

$$\begin{bmatrix} & \color{red}{y} & \color{blue}{g} & \color{green}{m} \\ \color{red}{y} & 24.30 &-6.70 & 6.85\\ \color{blue}{g} & -6.70 & 7.30 & -2.15 \\ \color{green}{m} & 6.85 & -2.15 & 3.70 \\ \end{bmatrix}$$

So it went from the $5 \times 3$ $A$ matrix to a $3 \times 3$ matrix.

The operations involved in calculating the correlation matrix are similar, but the data points are standardized by dividing each one by the standard deviation of the returns of each company (column vectors), right after centering the data points by subtracting the column means as in the covariance matrix:

$$\small cor(A)=\tiny\frac{1}{n-1}\small\begin{bmatrix} \frac{\color{red}{y_1 - \bar{y}}}{\color{red}{sd(y)}} & \frac{\color{red}{y_2 - \bar{y}}}{\color{red}{sd(y)}} & \frac{\color{red}{y_3 - \bar{y}}}{\color{red}{sd(y)}} & \frac{\color{red}{y_4 - \bar{y}}}{\color{red}{sd(y)}} &\frac{\color{red}{y_5 - \bar{y}}}{\color{red}{sd(y)}} \\ \frac{\color{blue}{g_1 - \bar{g}}}{\color{blue}{sd(g)}} & \frac{\color{blue}{g_2 - \bar{g}}}{\color{blue}{sd(g)}} & \frac{\color{blue}{g_3 - \bar{g}}}{\color{blue}{sd(g)}} & \frac{\color{blue}{g_4 - \bar{g}}} {\color{blue}{sd(g)}}& \frac{\color{blue}{g_5 - \bar{g}}}{\color{blue}{sd(g)}}\\ \frac{\color{green}{m_1 - \bar{m}}}{\color{green}{sd(m)}}& \frac{\color{green}{m_2 - \bar{m}}}{\color{green}{sd(m)}} &\frac{\color{green}{m_3 - \bar{m}}}{\color{green}{sd(m)}} & \frac{\color{green}{m_4 - \bar{m}}}{\color{green}{sd(m)}} & \frac{\color{green}{m_5 - \bar{m}}}{\color{green}{sd(m)}}\\ &&\color{purple} {3\times 5 \,\text{matrix}} \end{bmatrix} \begin{bmatrix} \frac{\color{red}{y_1 - \bar{y}}}{\color{red}{sd(y)}} & \frac{\color{blue}{g_1 - \bar{g}}}{\color{blue}{sd(g)}} & \frac{\color{green}{m_1 - \bar{m}}}{\color{green}{sd(m)}} \\ \frac{\color{red}{y_2 - \bar{y}}}{\color{red}{sd(y)}} & \frac{\color{blue}{g_2 - \bar{g}}}{\color{blue}{sd(g)}} & \frac{\color{green}{m_2 - \bar{m}}}{\color{green}{sd(m)}} \\ \frac{\color{red}{y_3 - \bar{y}}}{\color{red}{sd(y)}} &\frac{\color{blue}{g_3 - \bar{g}}}{\color{blue}{sd(g)}} & \frac{\color{green}{m_3 - \bar{m}}}{\color{green}{sd(m)}} \\ \frac{\color{red}{y_4 - \bar{y}}}{\color{red}{sd(y)}} & \frac{\color{blue}{g_4 - \bar{go}}}{\color{blue}{sd(g)}} & \frac{\color{green}{m_4 - \bar{m}}}{\color{green}{sd(m)}} \\ \frac{\color{red}{y_5 - \bar{y}}}{\color{red}{sd(y)}} & \frac{\color{blue}{g_5 - \bar{g}}}{\color{blue}{sd(g)}} & \frac{\color{green}{m_5 - \bar{m}}}{\color{green}{sd(m)}} \\ &\color{purple} {5\times 3 \,\text{matrix}} \end{bmatrix}$$

One more quick thing for completeness sake: We have so far a clunky matrix as the result, but in general we want to estimate the portfolio variance: $1$ portfolio; $1$ variance. To do that we multiply the matrix of variance-covariance of $A$ to the left and to the right by the vector containing the proportions or weightings in each stock - $W$. Since we want to end up with a scalar single number, it is unsurprising that the algebra will be: $W^T\,\sigma(A)\,W$, with the vector of weights (fractions) being in this case $\color{blue}{3}\times \color{blue}{1}$ to match perfectly on the left as $W^T$, and on the right as $W$.

Code in R:

Fictitious data set of returns in billions, percentage (?) - the matrix A:

yah = c(1, - 4, 5, 7, 8)
goog = c(8, 9, 10, 3, 7)
ms = c(1, 3, 4, 5, 6)
returns <- cbind(yah, goog, ms)
row.names(returns) =c("Yr.1","Yr.2","Yr.3","Yr.4", "Yr.5")


Centered matrix (G) of demeaned returns:

demeaned_returns <- scale(returns, scale = F, center = T)


Manual and R function calculation of the variance-covariance matrix:

(var_cov_A = (t(demeaned_returns)%*%demeaned_returns)/(nrow(returns)-1))
cov(returns)   # the R in-built function cov() returns the same results.


Correlation matrix calculation:

We need to divide by the standard deviation column-wise:

demeaned_scaled_returns <- scale(returns, scale = T, center = T)


and then proceed as above:

(corr_A = (t(demeaned_scaled_returns) %*% demeaned_scaled_returns)/(nrow(returns)-1))
cor(returns) # Again, the R function returns the same matrix.

• Do you mean $G^{T}G$ instead? I did $G^{T}G$ but everything larger than what R returns by a factor of $8$. Feb 1 '16 at 17:47
• Now it's corrected. The idea is to match the number of rows in the first matrix that you are multiplying, with the number of columns in the second matrix. Feb 1 '16 at 17:52
• $G^\top G$ is not a correlation matrix. You need to divide by variances and also by $n$. Feb 1 '16 at 17:53
• That is my problem essentially, I don't know how to divide the variance in a computing package like R because of non-singular matrix problems, which arise from trying to invert the said matrix. Feb 1 '16 at 19:10
• I included the code to calculate the correlation matrix. No problems with inverting. I thought that your initial question had more to do with dimensions of the matrix. Hope it helps... Feb 1 '16 at 19:57