Variance of projected data

Question

What's the expression for the variance of projection of $N$ data points onto a vector $\mathbf{v}$ which is not necessarily a unit vector? I could only find an expression when $\mathbf{v}$ is taken to be a unit vector.

Here's my attempt

Let's say our data is represented by a matrix $\mathbf{X} \in \mathbb{R}^{N \times p}$ and is already centered, i.e. $\mathbf{X} = \begin{bmatrix} (\mathbf{x}_1 - \bar{\mathbf{x}})^\intercal \\ \vdots \\ (\mathbf{x}_N - \bar{\mathbf{x}})^\intercal \end{bmatrix}$, where $\bar{\mathbf{x}} = \frac{1}{N} \sum_{i=1}^N \mathbf{x}_i$.

The projection for each data point is $\frac{\mathbf{x}_i^\intercal\mathbf{v}}{\mathbf{v}^\intercal\mathbf{v}}$. The variance can then be computed as follows:

$$ \begin{aligned} \hat{\sigma}^2 &= \frac{1}{N} \sum_{i=1}^N \Big(\frac{\mathbf{x}_i^\intercal\mathbf{v}}{\mathbf{v}^\intercal\mathbf{v}} - \frac{\bar{\mathbf{x}}^\intercal\mathbf{v}}{\mathbf{v}^\intercal\mathbf{v}} \Big)^2 \\ &= \frac{1}{N} \frac{1}{(\mathbf{v}^\intercal\mathbf{v})^2}\big(\mathbf{X}\mathbf{v}\big)^\intercal\big(\mathbf{X}\mathbf{v}\big) \\ &= \frac{1}{N} \frac{\mathbf{v}^\intercal\mathbf{X}^\intercal\mathbf{X}\mathbf{v}}{(\mathbf{v}^\intercal\mathbf{v})^2} \\ &= \frac{\mathbf{v}^\intercal \mathbf{S}\mathbf{v}}{(\mathbf{v}^\intercal\mathbf{v})^2} \end{aligned} $$

where $\mathbf{S}$ is the covariance matrix of the original data.

Is this correct? I am using it as an intermediate step in another problem and the expression I derived above for variance of projected data doesn't seem to be working.

Answer 1

The projection of $x$ onto the span of $v$ is a vector $\frac{x^\top v}{v^\top v} v$, but this vector has length $\frac{x^\top v}{v^\top v} \|v\| = \frac{x^\top v}{\sqrt{v^\top v}}$, which is what you should be using in your definition of variance. This will lead to $\frac{v^\top S v}{v^\top v}$ as the expression for variance (notice that replacing $v$ by $2v$ would not change this expression, but your expression $\frac{v^\top S v}{(v^\top v)^2}$ would get halved), and will lead to the desired result in your other post, as you noted in the comment.