Given the positive semi-definite, symmetric matrix $A = bb^T + \sigma^2I$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know that it has real eigenvalues since it's symmetric and positive semidefinite but not sure about solving directly for those values and their corresponding vectors.

  • 1
    $\begingroup$ What is $b$? Would it be a column vector, a rectangular matrix, or a square matrix? Since the singular values are well defined mathematically and loads of software exists to find them, please explain what you mean by "is it possible to find." $\endgroup$ – whuber Nov 1 '20 at 16:50
  • 1
    $\begingroup$ $b$ is a column vector and I mean an analytical solution $\endgroup$ – user301437 Nov 1 '20 at 19:12

The singular values are the eigenvalues of $A.$ By definition, when there exists a nonzero vector $\mathbf x$ for which $A\mathbf{x}=\lambda \mathbf{x},$ $\lambda$ is an eigenvalue and $\mathbf{x}$ is a corresponding eigenvector.

Note, then, that

$$A\mathbf{b} = (\mathbf{b}\mathbf{b}^\prime + \sigma^2I)\mathbf{b} = \mathbf{b}(\mathbf{b}^\prime \mathbf{b}) + \sigma^2 \mathbf{b} = (|\mathbf{b}|^2+\sigma^2)\mathbf{b},$$

demonstrating that $\mathbf{b}$ is an eigenvector with eigenvalue $\lambda_1 = |\mathbf{b}|^2 + \sigma^2.$

Furthermore, whenever $\mathbf{x}$ is a vector orthogonal to $\mathbf{b}$ -- that is, when $\mathbf{b}^\prime \mathbf{x} = \pmatrix{0},$ we may similarly compute

$$A\mathbf{x} = (\mathbf{b}\mathbf{b}^\prime + \sigma^2I)\mathbf{x} = \mathbf{b}(\mathbf{b}^\prime \mathbf{x}) + \sigma^2 \mathbf{x} = (0+\sigma^2)\mathbf{x},$$

showing that all such vectors are eigenvectors with eigenvalue $\sigma^2.$

Provided these vectors are in a finite dimensional vector space of dimension $n$ (say), a straightforward induction establishes that the vectors $x$ for which $\mathbf{b}^\prime \mathbf{x}=0$ form a subspace $\mathbf{b}^\perp$ of dimension $n-1.$ Let $\mathbf{e}_2, \ldots, \mathbf{e}_n$ be an orthonormal basis for this subspace. It extends to an orthonormal basis $\mathscr{E} = (\mathbf{\hat b}, \mathbf{e}_2, \ldots, \mathbf{e}_n)$ of the whole space where $\mathbf{\hat b} = \mathbf{b}/|\mathbf{b}|$. In terms of this basis the matrix of $A$ therefore is

$$\operatorname{Mat}(A, \mathscr{E}, \mathscr{E}) = \pmatrix{|\mathbf{b}|^2+\sigma^2 & 0 & 0 & \cdots & 0 \\ 0 & \sigma^2 & 0 & \cdots & 0 \\ 0 & 0 & \ddots & \vdots & \vdots \\ \vdots & \vdots & \cdots & \ddots & 0 \\ 0 & 0 & \cdots & 0 & \sigma^2 }$$

Whether or not every step of this derivation was clear, you can verify the result by setting

$$Q = \left(\mathbf{b}; \mathbf{e}_2; \ldots; \mathbf{e}_n\right)$$

to be the matrix with the the given columns and computing

$$Q\,\operatorname{Mat}(A, \mathscr{E}, \mathscr{E})\,Q^\prime = \mathbf{b}^\prime + \sigma^2I = A.$$

This is explicitly a singular value decomposition of the form $U\Sigma V^\prime$ where $V=Q,$ $\Sigma= \operatorname{Mat}(A, \mathscr{E}, \mathscr{E}),$ and $U=Q^\prime.$

The Gram Schmidt process provides a general algorithm to find $\mathscr{E}$ (and therefore $Q$): its input is the series of vectors $\mathbf{\hat b}$, $(1,0,\ldots,0)^\prime,$ and so on through $(0,\ldots,0,1)^\prime.$ After $n-1$ steps it will produce an orthonormal basis including the starting vector $\mathbf b.$

As an example, let $\mathbf{b} = (3,4,0)^\prime.$ With $\sigma^2 = 1,$ compute

$$\mathbf{b}\mathbf{b}^\prime + \sigma^2 I = \pmatrix{10&12&0\\12&17&0\\0&0&1}$$

Here, $|\mathbf{b}|^2 = 3^2+4^2+0^2=5^2,$ so that $\mathbf{\hat b} = \mathbf{b}/5 = (3/5,4/5,0)^\prime.$ One way to extend this to an orthonormal basis is to pick $\mathbf{e}_2 = (-4/5,3/5,0)^\prime$ and $\mathbf{e}_3 = (0,0,1)^\prime.$ Thus

$$Q = \pmatrix{3/5&4/5&0\\-4/5&3/5&0\\0&0&1}$$

and we may confirm that

$$\begin{align} Q\,\operatorname{Mat}(A, \mathscr{E}, \mathscr{E})\,Q^\prime &= \pmatrix{3/5&4/5&0\\-4/5&3/5&0\\0&0&1}\pmatrix{5^2+1^2&0&0\\0&1&0\\0&0&1}\pmatrix{3/5&-4/5&0\\4/5&3/5&0\\0&0&1}\\ &=\pmatrix{10&12&0\\12&17&0\\0&0&1} \end{align}$$

as intended.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy