Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a matrix $G$ in which I would like to compute the correlation matrix of. The matrix in R looks like:

> G
     [,1] [,2] [,3] [,4] [,5] [,6]
[1,]   -1    0   -1   -1    0   -1
[2,]   -1    0   -1    1    1    0
[3,]   -1    0    1   -1    0    1
[4,]   -1    0    1    1   -1    0
[5,]    1   -1    0   -1    0    1
[6,]    1   -1    0    1   -1    0
[7,]    1    1    0   -1    0   -1
[8,]    1    1    0    1    1    0

Now, I would like to find the correlation matrix. However, when I use the function cor(), I get the following:

> cor(G)
     [,1] [,2] [,3] [,4] [,5] [,6]
[1,]    1  0.0  0.0    0  0.0  0.0
[2,]    0  1.0  0.0    0  0.5 -0.5
[3,]    0  0.0  1.0    0 -0.5  0.5
[4,]    0  0.0  0.0    1  0.0  0.0
[5,]    0  0.5 -0.5    0  1.0  0.0
[6,]    0 -0.5  0.5    0  0.0  1.0

Why were two rows omitted here? Am I missing something?

share|improve this question
4  
To whom it may concern, it seems to me this is really a statistical understanding issue, even though it is couched in terms of R. IMO, this should be considered on topic here. – gung Feb 1 at 18:02

Correlation is calculated between columns and not between rows.

The output should be read as, correlation between column-i and column j.

Since you have 6 columns, you get a 6x6 correlation matrix. All 8 rows have been considered while calculating these correlations.

share|improve this answer

In the $G^TG$ operation, $G^T$ is an $6 \times 8$ matrix, and $G$ an $8 \times 6$. Hence, the matrix multiplication will yield a $6 \times 6$ matrix.

Addressing the comments and the underlying issue, let's pretend that we have a matrix corresponding to returns of different stocks (in the columns) versus 5 consecutive years (in the rows) - completely fictitious stocks and years:

$$\begin{bmatrix} & yah(y) & gog(g) & ms(m) \\ Yr.1 & 1 &8 & 1\\ Yr.2 & -4 &9 & 3 \\ Yr.3 & 5 & 10 & 4 \\ Yr.4 & 7 & 3 & 5\\ Yr.5 & 8 & 7& 6 \end{bmatrix}$$

Let's call this matrix $A$. We want to calculate the correlations between the different vectors of returns, one for each company, "packaged" in the matrix $A$.

The variance-covariance matrix of the portfolio assuming equal holdings will be:

$\Large \sigma(A) = \frac{G^TG}{n-1}$ with $G$ being the mean-centered observations and $n-1$ corresponding to the number of observations minus $1$.

The mean-centered (or demeaned) matrix $G$ is:

$$\begin{bmatrix} & y & g & m \\ Yr.1 & -2.4 &0.6 & -2.8\\ Yr.2 & -7.4 &1.6 & -0.8 \\ Yr.3 & 1.6 & 2.6 & 0.2 \\ Yr.4 & 3.6 & -4.4 & 1.2\\ Yr.5 & 4.6 & -0.4& 2.2 \end{bmatrix} $$

And the variance-covariance matrix:

$$\begin{bmatrix} & y & g & m \\ y & 24.30 &-6.70 & 6.85\\ g & -6.70 & 7.30 & -2.15 \\ ms & 6.85 & -2.15 & 3.70 \\ \end{bmatrix} $$

So it went from the $5 \times 3$ $A$ matrix to a $3 \times 3$ matrix.

The operations involved in calculating the correlation matrix are similar, but the data points are standardized by dividing each one by the standard deviation of the returns of each company (column vectors), right after centering the data points by subtracting the column means as in the covariance matrix:

$$\small cor(A)=\tiny\frac{1}{n-1}\small\begin{bmatrix} \frac{y_1 - \bar{a}}{sd(ya)} & \frac{y_2 - \bar{y}}{sd(y)} & \frac{y_3 - \bar{y}}{sd(y)} & \frac{y_4 - \bar{y}}{sd(ya)} &\frac{y_5 - \bar{y}}{sd(y)} \\ \frac{g_1 - \bar{g}}{sd(g)} & \frac{g_2 - \bar{g}}{sd(g)} & \frac{g_3 - \bar{g}}{sd(g)} & \frac{g_4 - \bar{g}} {sd(g)}& \frac{g_5 - \bar{g}}{sd(g)}\\ \frac{m_1 - \bar{m}}{sd(m)}& \frac{m_2 - \bar{m}}{sd(m)} &\frac{m_3 - \bar{m}}{sd(m)} & \frac{m_4 - \bar{m}}{sd(m)} & \frac{m_5 - \bar{m}}{sd(m)}\\ &&\color{blue} {3\times 5 \,\text{matrix}} \end{bmatrix} \begin{bmatrix} \frac{y_1 - \bar{y}}{sd(y)} & \frac{g_1 - \bar{g}}{sd(g)} & \frac{m_1 - \bar{m}}{sd(m)} \\ \frac{y_2 - \bar{y}}{sd(y)} & \frac{g_2 - \bar{g}}{sd(g)} & \frac{m_2 - \bar{m}}{sd(m)} \\ \frac{y_3 - \bar{y}}{sd(y)} &\frac{g_3 - \bar{g}}{sd(g)} & \frac{m_3 - \bar{m}}{sd(m)} \\ \frac{y_4 - \bar{y}}{sd(y)} & \frac{g_4 - \bar{go}}{sd(g)} & \frac{m_4 - \bar{m}}{sd(m)} \\ \frac{y_5 - \bar{y}}{sd(y)} & \frac{g_5 - \bar{g}}{sd(g)} & \frac{m_5 - \bar{m}}{sd(m)} \\ &\color{blue} {5\times 3 \,\text{matrix}} \end{bmatrix}$$

One more quick thing for completeness sake: We have so far a clunky matrix as the result, but in general we want to estimate the portfolio variance - 1 portfolio; 1 variance. To do that we multiply the matrix of variance-covariance of $A$ to the left and to the right by the vector containing the proportions or weightings in each stock - $W$. Since we want to end up with a scalar single number, it is unsurprising that the algebra will be: $W^T\,\sigma(A)\,W$, with the vector of weights (fractions) being in this case $\color{blue}{3}\times \color{blue}{1}$ to match perfectly on the left as $W^T$, and on the right as $W$.

Code in R:

Fictitious data set of returns in billions, percentage (?) - the matrix A:

yah = c(1, - 4, 5, 7, 8)
goog = c(8, 9, 10, 3, 7)
ms = c(1, 3, 4, 5, 6)
returns <- cbind(yah, goog, ms)
row.names(returns) =c("Yr.1","Yr.2","Yr.3","Yr.4", "Yr.5")

Centered matrix (G) of demeaned returns:

demeaned_returns <- scale(returns, scale = F, center = T)

Manual and R function calculation of the variance-covariance matrix:

(var_cov_A = (t(demeaned_returns)%*%demeaned_returns)/(nrow(returns)-1))
cov(returns)   # the R in-built function cov() returns the same results.

Correlation matrix calculation:

We need to divide by the standard deviation column-wise:

demeaned_scaled_returns <- scale(returns, scale = T, center = T)

and then proceed as above:

(corr_A = (t(demeaned_scaled_returns) %*% demeaned_scaled_returns)/(nrow(returns)-1))
cor(returns) # Again, the R function returns the same matrix.
share|improve this answer
    
Do you mean $G^{T}G$ instead? I did $G^{T}G$ but everything larger than what R returns by a factor of $8$. – user1398057 Feb 1 at 17:47
    
Now it's corrected. The idea is to match the number of rows in the first matrix that you are multiplying, with the number of columns in the second matrix. – Antoni Parellada Feb 1 at 17:52
1  
$G^\top G$ is not a correlation matrix. You need to divide by variances and also by $n$. – amoeba Feb 1 at 17:53
    
That is my problem essentially, I don't know how to divide the variance in a computing package like R because of non-singular matrix problems, which arise from trying to invert the said matrix. – user1398057 Feb 1 at 19:10
    
I included the code to calculate the correlation matrix. No problems with inverting. I thought that your initial question had more to do with dimensions of the matrix. Hope it helps... – Antoni Parellada Feb 1 at 19:57

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.