Let $X$ be a random vector of dimension $p$ and $\{ X_1, \dots, X_n \}$ the $n$ observations of such vector. Let $\mathbb{X}$ be the matrix with rows $X_k$ and $\mathbb{X}_c$ is the matrix with rows $X_k - \mu_X$, where $\mu_X$ is the empirical mean of $X$, $\mu_X = \sum_{k=1}^n X_k $.

In class, the question was if the first eigenvector of $\mathbb{X}^t \mathbb{X}$ and $\mathbb{X}_c^t \mathbb{X}_c$ are different. The answer is yes and we saw an example like this: enter image description here

Where the blue segment is the first eigenvector of the centered points $\mathbb{X}_c^t \mathbb{X}_c$ and the black segment is the first eigenvector of the points without centering $\mathbb{X}^t \mathbb{X}$, this vector seems to point to the center of $X$.

I know that the matrix $\mathbb{X}_c^t \mathbb{X}_c$ is proportional to the covariance matrix $Cov(X)=\frac{1}{n}\mathbb{X}_c^t \mathbb{X}_c$, then they have the same eigenvectors. The first eigenvector of $Cov(X)$ is the direction of the first principal component.

With some algebra I obtain the following relation between $Cov(X)$ and $\mathbb{X}^t \mathbb{X}$: $$ Cov(X) = \frac{1}{n} \mathbb{X}^t \mathbb{X} - \mu_X \mu_X^t $$

The first eigenvalue of $\mathbb{X}^t \mathbb{X}$ is the vector $v$ that maximizes $v^t \mathbb{X}^t \mathbb{X} v $, then $v$ also maximizes: $$v^t ( Cov(X) + \mu_X \mu_X^t ) v $$

My question is if there is an expression of the first eigenvector of $\mathbb{X}^t \mathbb{X}$ that could tell me how this vector is going to look geometrically.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.