1
$\begingroup$

Let $X$ be a random vector of dimension $p$ and $\{ X_1, \dots, X_n \}$ the $n$ observations of such vector. Let $\mathbb{X}$ be the matrix with rows $X_k$ and $\mathbb{X}_c$ is the matrix with rows $X_k - \mu_X$, where $\mu_X$ is the empirical mean of $X$, $\mu_X = \sum_{k=1}^n X_k $.

In class, the question was if the first eigenvector of $\mathbb{X}^t \mathbb{X}$ and $\mathbb{X}_c^t \mathbb{X}_c$ are different. The answer is yes and we saw an example like this: enter image description here

Where the blue segment is the first eigenvector of the centered points $\mathbb{X}_c^t \mathbb{X}_c$ and the black segment is the first eigenvector of the points without centering $\mathbb{X}^t \mathbb{X}$, this vector seems to point to the center of $X$.

I know that the matrix $\mathbb{X}_c^t \mathbb{X}_c$ is proportional to the covariance matrix $Cov(X)=\frac{1}{n}\mathbb{X}_c^t \mathbb{X}_c$, then they have the same eigenvectors. The first eigenvector of $Cov(X)$ is the direction of the first principal component.

With some algebra I obtain the following relation between $Cov(X)$ and $\mathbb{X}^t \mathbb{X}$: $$ Cov(X) = \frac{1}{n} \mathbb{X}^t \mathbb{X} - \mu_X \mu_X^t $$

The first eigenvalue of $\mathbb{X}^t \mathbb{X}$ is the vector $v$ that maximizes $v^t \mathbb{X}^t \mathbb{X} v $, then $v$ also maximizes: $$v^t ( Cov(X) + \mu_X \mu_X^t ) v $$

My question is if there is an expression of the first eigenvector of $\mathbb{X}^t \mathbb{X}$ that could tell me how this vector is going to look geometrically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.