# Largest singular values

Given the positive semi-definite, symmetric matrix $$A = bb^T + \sigma^2I$$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know that it has real eigenvalues since it's symmetric and positive semidefinite but not sure about solving directly for those values and their corresponding vectors.

• What is $b$? Would it be a column vector, a rectangular matrix, or a square matrix? Since the singular values are well defined mathematically and loads of software exists to find them, please explain what you mean by "is it possible to find." – whuber Nov 1 '20 at 16:50
• $b$ is a column vector and I mean an analytical solution – user301437 Nov 1 '20 at 19:12

The singular values are the eigenvalues of $$A.$$ By definition, when there exists a nonzero vector $$\mathbf x$$ for which $$A\mathbf{x}=\lambda \mathbf{x},$$ $$\lambda$$ is an eigenvalue and $$\mathbf{x}$$ is a corresponding eigenvector.

Note, then, that

$$A\mathbf{b} = (\mathbf{b}\mathbf{b}^\prime + \sigma^2I)\mathbf{b} = \mathbf{b}(\mathbf{b}^\prime \mathbf{b}) + \sigma^2 \mathbf{b} = (|\mathbf{b}|^2+\sigma^2)\mathbf{b},$$

demonstrating that $$\mathbf{b}$$ is an eigenvector with eigenvalue $$\lambda_1 = |\mathbf{b}|^2 + \sigma^2.$$

Furthermore, whenever $$\mathbf{x}$$ is a vector orthogonal to $$\mathbf{b}$$ -- that is, when $$\mathbf{b}^\prime \mathbf{x} = \pmatrix{0},$$ we may similarly compute

$$A\mathbf{x} = (\mathbf{b}\mathbf{b}^\prime + \sigma^2I)\mathbf{x} = \mathbf{b}(\mathbf{b}^\prime \mathbf{x}) + \sigma^2 \mathbf{x} = (0+\sigma^2)\mathbf{x},$$

showing that all such vectors are eigenvectors with eigenvalue $$\sigma^2.$$

Provided these vectors are in a finite dimensional vector space of dimension $$n$$ (say), a straightforward induction establishes that the vectors $$x$$ for which $$\mathbf{b}^\prime \mathbf{x}=0$$ form a subspace $$\mathbf{b}^\perp$$ of dimension $$n-1.$$ Let $$\mathbf{e}_2, \ldots, \mathbf{e}_n$$ be an orthonormal basis for this subspace. It extends to an orthonormal basis $$\mathscr{E} = (\mathbf{\hat b}, \mathbf{e}_2, \ldots, \mathbf{e}_n)$$ of the whole space where $$\mathbf{\hat b} = \mathbf{b}/|\mathbf{b}|$$. In terms of this basis the matrix of $$A$$ therefore is

$$\operatorname{Mat}(A, \mathscr{E}, \mathscr{E}) = \pmatrix{|\mathbf{b}|^2+\sigma^2 & 0 & 0 & \cdots & 0 \\ 0 & \sigma^2 & 0 & \cdots & 0 \\ 0 & 0 & \ddots & \vdots & \vdots \\ \vdots & \vdots & \cdots & \ddots & 0 \\ 0 & 0 & \cdots & 0 & \sigma^2 }$$

Whether or not every step of this derivation was clear, you can verify the result by setting

$$Q = \left(\mathbf{b}; \mathbf{e}_2; \ldots; \mathbf{e}_n\right)$$

to be the matrix with the the given columns and computing

$$Q\,\operatorname{Mat}(A, \mathscr{E}, \mathscr{E})\,Q^\prime = \mathbf{b}^\prime + \sigma^2I = A.$$

This is explicitly a singular value decomposition of the form $$U\Sigma V^\prime$$ where $$V=Q,$$ $$\Sigma= \operatorname{Mat}(A, \mathscr{E}, \mathscr{E}),$$ and $$U=Q^\prime.$$

The Gram Schmidt process provides a general algorithm to find $$\mathscr{E}$$ (and therefore $$Q$$): its input is the series of vectors $$\mathbf{\hat b}$$, $$(1,0,\ldots,0)^\prime,$$ and so on through $$(0,\ldots,0,1)^\prime.$$ After $$n-1$$ steps it will produce an orthonormal basis including the starting vector $$\mathbf b.$$

As an example, let $$\mathbf{b} = (3,4,0)^\prime.$$ With $$\sigma^2 = 1,$$ compute

$$\mathbf{b}\mathbf{b}^\prime + \sigma^2 I = \pmatrix{10&12&0\\12&17&0\\0&0&1}$$

Here, $$|\mathbf{b}|^2 = 3^2+4^2+0^2=5^2,$$ so that $$\mathbf{\hat b} = \mathbf{b}/5 = (3/5,4/5,0)^\prime.$$ One way to extend this to an orthonormal basis is to pick $$\mathbf{e}_2 = (-4/5,3/5,0)^\prime$$ and $$\mathbf{e}_3 = (0,0,1)^\prime.$$ Thus

$$Q = \pmatrix{3/5&4/5&0\\-4/5&3/5&0\\0&0&1}$$

and we may confirm that

\begin{align} Q\,\operatorname{Mat}(A, \mathscr{E}, \mathscr{E})\,Q^\prime &= \pmatrix{3/5&4/5&0\\-4/5&3/5&0\\0&0&1}\pmatrix{5^2+1^2&0&0\\0&1&0\\0&0&1}\pmatrix{3/5&-4/5&0\\4/5&3/5&0\\0&0&1}\\ &=\pmatrix{10&12&0\\12&17&0\\0&0&1} \end{align}

as intended.