Change in eigenvalues due to perturbation to a correlation matrix - Cross Validated most recent 30 from stats.stackexchange.com 2019-10-21T15:48:53Z https://stats.stackexchange.com/feeds/question/381129 https://creativecommons.org/licenses/by-sa/4.0/rdf https://stats.stackexchange.com/q/381129 1 Change in eigenvalues due to perturbation to a correlation matrix hari https://stats.stackexchange.com/users/229700 2018-12-09T17:36:45Z 2018-12-10T01:51:08Z <p>Let <span class="math-container">$A$</span> be a <span class="math-container">$m \times n$</span> matrix defined as <span class="math-container">$A = \Big[\frac{a_1}{\|a_1\|} \cdots \frac{a_n}{\|a_n\|}\Big]$</span> and <span class="math-container">$a_k \in \mathbb{R}^{m\times 1}$</span> where <span class="math-container">$k \in [1,\dots,n]$</span>.</p> <p>Now, we define a correlation matrix <span class="math-container">$R = A^TA$</span> where each diagonal element is <span class="math-container">$1$</span> and it is a symmetric matrix. The trace of <span class="math-container">$R$</span>, i.e., <span class="math-container">$\mathbb{Tr}(R) = n$</span>.</p> <p>All non-diagonal elements of <span class="math-container">$R$</span> represents the correlation among the columns of <span class="math-container">$A$</span>. We define them by correlation-coefficients <span class="math-container">$\rho_{jk} = \Big(\frac{a_j}{\|a_j\|}\Big)^T\Big(\frac{a_k}{\|a_k\|}\Big)$</span> which satisfy <span class="math-container">$-1 \leq \rho_{jk} \leq 1$</span>. </p> <p>In my present work, I modify each column of <span class="math-container">$A$</span> such that the correlation among the columns of <span class="math-container">$A$</span> increases. Consequently, the correlation-coefficients also increases proportionally in <span class="math-container">$R$</span>. I am interested to comment on the change in eigenvalues of <span class="math-container">$R$</span> with increase in correlation-coefficients. </p> <p>Numerically, I observed that only largest eigenvalue of <span class="math-container">$R$</span> increases whereas rest of the eigenvalues decreases. But, I am unable to verify this phenomenon theoretically. Therefore, I ask you here for a hint to proceed my investigation further.</p> <p>More precisely, the claim is:</p> <p>Let <span class="math-container">$\lambda$</span> be the set of eigenvalues of <span class="math-container">$R$</span> where <span class="math-container">$\lambda_1 \geq \cdots \geq \lambda_n \geq 0$</span> and <span class="math-container">$\hat{\lambda}$</span> be the set of eigenvalues of <span class="math-container">$\hat{R}$</span> where <span class="math-container">$\hat{\lambda}_1 \geq \cdots \geq \hat{\lambda}_n \geq 0$</span>. Assume that the correlation-coefficients in <span class="math-container">$\hat{R}$</span> satisfy <span class="math-container">$$\hat{\rho}_{jk} \geq \rho_{jk} \quad {j,k} \in [1,\cdots,n] \quad \text{and} \quad j\neq k.$$</span> Moreover the trace of correlation matrices remains same, i.e., <span class="math-container">$$\mathbb{Tr}(\hat{R}) = \mathbb{Tr}({R}) = n.$$</span> Consequently, we claim that the eigenvalues of <span class="math-container">$\hat{R}$</span> and <span class="math-container">$R$</span> satisfy the following inequalities: <span class="math-container">$$\hat{\lambda}_1 \geq \lambda_1 \quad \text{and} \quad \hat{\lambda}_i \leq \lambda_i \quad i\in[2,\dots,n].$$</span></p> <p>Example:</p> <p>Suppose, all columns of <span class="math-container">$A$</span> are orthonormal. This implies that the resulting correlation matrix would be an identity matrix and in this case, all eigenvalues are equal to <span class="math-container">$1$</span>.</p> <p>Now, suppose all columns are linearly dependent by a positive factor. This implies that the all correlation-coefficients is equal to 1 and the resulting correlation matrix is a rank-1 matrix, i.e., <span class="math-container">$\mathbb{1}\mathbb{1}^T$</span> where the largest eigenvalue is <span class="math-container">$n$</span> and rest of the eigenvalues are zero.</p> <p>In this example, the largest eigenvalue increases from <span class="math-container">$1$</span> to <span class="math-container">$n$</span> when correlation matrix changes from the identity matrix to the rank-1 matrix. On the other hand, rest of the eigenvalues decreases from <span class="math-container">$1$</span> to <span class="math-container">$0$</span>. </p> <p>In order to prove the above mentioned claim, will it be sufficient:</p> <p>if we can show that the largest eigenvalue path from the identity matrix to a matrix of ones, i.e., <span class="math-container">$\mathbb{1}\mathbb{1}^T$</span> is monotonically non-decreasing. Here, we will change only off-diagonal elements which always lies between -1 to 1. We also establish similar behaviour for rest of the eigenvalues ?</p> <p>Can you comment on this approach? If you think, it could be a right direction. Do you have any suggestion how should I start to prove/disprove the claim?</p>