3
$\begingroup$

Suppose $\mathbf{b}=[b_1,b_2]'$ is $2\times 1$ and $\mathbf{C}$ is a full-rank symmetric $2\times 2$ matrix which both are real and given. Now, consider the problem of finding a $2\times 2$ matrix $\mathbf{A}$ such that $\mathbf{b'}=\mathbf{A}\mathbf{b}$ is in the same direction of one of the eigenvectors of $\mathbf{A}\mathbf{C}\mathbf{A}^T$ (lets say the eigenvector correspodning to the bigger eigenvalue).

I have been looking a lot to find a solution for $\mathbf{A}$ through formulating the problem as an optimization problem. However, I have not been able to see what type of optimization problem this falls into. I appreciate it if you could introduce me to some references addressing a similar problem to this one, or outline the appropriate approach for solving such examples.

Improve this question
$\endgroup$
6
  • $\begingroup$ If $A$ is not further constrained, you can always choose $A$ so that $ACA^T = I$ (e.g. as here). In that case the covariance is isotropic/spherical, so the direction of $Ab$ will not matter. Can you expand on your goal? $\endgroup$
    – GeoMatt22
    Commented May 29, 2020 at 16:24
  • $\begingroup$ Would $C$ happen to be symmetric? $\endgroup$
    – whuber ♦
    Commented May 29, 2020 at 21:42
  • $\begingroup$ @whuber yes, $\mathbf{C}$ is symmetric. $\endgroup$
    – nOp
    Commented May 30, 2020 at 0:05
  • $\begingroup$ @GeoMatt22 Based on your argument, clearly, the $\mathbf{A}$ you introduced is not what we are after. We want an $\mathbf{A}$ such that $\mathbf{A}b$ points in a certain direction. $\endgroup$
    – nOp
    Commented May 30, 2020 at 0:09
  • $\begingroup$ From your title, I was inferring that $C$ is a covariance matrix. Then $\hat{C} = ACA^T$ is the transformed covariance. Both covariances can be visualized as ellipses, so the "direction" is along the major axis of the ellipse. I was pointing out that $A$ can be chosen so that $\hat{C}$ is a circle, i.e. with no preferred orientation. Did you intend to define the direction independently of $A$? $\endgroup$
    – GeoMatt22
    Commented May 30, 2020 at 0:32

1 Answer 1

Reset to default
3
$\begingroup$

Often, it's easiest (and insightful) to solve problems of linear algebra using algebraic or geometric methods rather than trying to cast them as optimization problems.

This meta-principle is justified by the fact that many linear algebraic problems, such as this one, make sense for arbitrary ground fields (and often for mere rings of scalars) but the optimization framework requires the ground field to be the real numbers or one of their field extensions, making optimization a (much) more limited technique.

Assume $b$ is nonzero (for otherwise, if we understand "in the same direction as" means "is a nonzero multiple of," there is no solution).

Pick any $\lambda\gt 0$ and let $A$ be a square root of $\lambda C^{-1};$ that is, choose $A$ such that $$A^\top A = \lambda C^{-1}.$$

(This strategy works very generally. In the present case, because $C$ is real symmetric, such a matrix $A$ exists and can be found with, say, a Cholesky decomposition.)

Since $C$ is of full rank so therefore is $C^{-1},$ whence $A$ must be invertible. Consequently

$$\left(ACA^\top\right)b^\prime= A\left(C^{-1}\right)^{-1}A^\top\,b^\prime = A\left(\lambda^{-1} A^\top A\right)^{-1}A^\top\,b^\prime = A \lambda A^{-1} \left(A^\top\right)^{-1} A^\top\,b^\prime = \lambda\, b^\prime$$

demonstrates every vector $b^\prime$ satisfies the eigenvector equation for $ACA^\top$ (with eigenvalue $\lambda$). Because $b^\prime = A b$ is nonzero if and only if $b$ is nonzero (due to the invertibility of $A$), $b^\prime$ is an eigenvector of $ACA^\top$ with eigenvalue $\lambda,$ QED.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks! Can anything be said regarding the part that we want $\mathbf{A}b$ to be in the direction of the biggest eigenvalue of $\mathbf{A}\mathbf{C}\mathbf{A}^T$? (for example for the case $2\times 2$ matrices) $\endgroup$
    – nOp
    Commented May 30, 2020 at 21:03
  • $\begingroup$ Since all eigenvalues of $ACA^\top$ are equal, automatically $Ab$ is an eigenvector with the largest eigenvalue. $\endgroup$
    – whuber ♦
    Commented May 31, 2020 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.