2
$\begingroup$

In this book on matrix factorizations, the author states the following, which I don't find to be true empirically. Is it true and under what conditions?

enter image description here

ADD: Trying to recreate the answer in R, what is the issue?

> D<-matrix(c(7,1,4,5,2,2,4,5,5,2,5,1,7,8,0,7),nrow=4,ncol=2,byrow=TRUE)
> D
     [,1] [,2]
[1,]    7    1
[2,]    4    5
[3,]    2    2
[4,]    4    5
> cor(D)
           [,1]       [,2]
[1,]  1.0000000 -0.3333333
[2,] -0.3333333  1.0000000
> 
> D[,1]<-D[,1]-mean(D[,1])
> D[,2]<-D[,2]-mean(D[,2])
> 
> D[,1]<-D[,1]/norm(as.matrix(D[,1]))
> D[,2]<-D[,2]/norm(as.matrix(D[,2]))
> D
            [,1]       [,2]
[1,]  0.50000000 -0.3214286
[2,] -0.04545455  0.2500000
[3,] -0.40909091 -0.1785714
[4,] -0.04545455  0.2500000
> t(D)%*%D
           [,1]       [,2]
[1,]  0.4214876 -0.1103896
[2,] -0.1103896  0.2602041
$\endgroup$
2
$\begingroup$

I think the author is simply handy-wavy there. I believe it is assumed that the columns of $A$ have norm 1 and mean 0.

$A^TA$ is a Gram matrix. Given you are using random variables to construct $A$, $A^TA$ is approximately proportional to the covariance matrix (scale by $n$ the number of variables). The correlation matrix is simply the scaled version of the covariance matrix. Clearly if your random variables in the columns of $A$ are already normalized to unit-norm then $A^TA$ does not need any further normalization and it is immediately a correlation matrix.

For example using MATLAB:

 % Set the seed and generate a random matrix
 rng(123);        
 A = randn(10,3);
 A'*A

 % ans =
 %  4.867589606955550  -3.004809945345502  -1.615373090426428
 % -3.004809945345502   5.356131932084330  -0.457643222208441
 % -1.615373090426428  -0.457643222208441   5.574303027408192.

 % Normalized now all the variables to have unit norm and mean zeros

 A = A - repmat(mean(A),10,1);
 A(:,1) = A(:,1)./norm(A(:,1));
 A(:,2) = A(:,2)./norm(A(:,2));
 A(:,3) = A(:,3)./norm(A(:,3));
 A'*A

 % ans =
 %  1.000000000000000  -0.568373671796724  -0.336996715690262
 % -0.568373671796724   1.000000000000000  -0.052272607974969
 % -0.336996715690262  -0.052272607974969   1.000000000000000

If we wanted to make the columns of $A$ orthogonal too, we would orthonormalize $A$. In that case we would do a Gram-Schmidt process on $A$; a process relating to a lot of wonderful things eg. Givens rotations, Householder transformation, etc.)

$\endgroup$
  • $\begingroup$ This is great, thank you!! I am curious if you program in R? I cant produce the correlation matrix following these steps and I wonder if my problem is software related? I added to the original question the code I was using. $\endgroup$ – B_Miner May 4 '15 at 13:15
  • 1
    $\begingroup$ I saw your code. norm(A(:,1)) in MATLAB is the Euclidean norm (norm-2). norm(as.matrix(D[,1])) in R is the maximum absolute norm (norm-1). You want to use: norm(as.matrix(D[,1]), '2'). More information can be found by typing ?norm in R terminal. $\endgroup$ – usεr11852 says Reinstate Monic May 5 '15 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.