Significance of eigenvector components in PCA Long time reader, first time poster. Hopefully I won't screw this up...
In the context of Principal Component Analysis, I have the sense that the components of an eigenvector are a measure of the contribution of each variable to the variation on that axes. I just can't find anything that puts it in those terms. Would someone confirm and provide support, or provide a more accurate understanding?
[ Edit: adding for clarity... ]
In other words, given the following expression for the first principle component (hope I've gotten this right from memory) ...
$$Y_1 = e_{11} Z_1 + e_{12}Z_2 + \cdots + e_{1p}Z_p$$
Is it reasonable to interpret, for example, $e_{11}$ as a measure of the contribution of $Z_1$ to the variance of the data the first component?
Now that I've written it out in this form the answer seems obvious as $Y_i$ is just a linear combination of $e_{ij}$ and the standardized data...
 A: Assume we are given $n$ random variables $X_1, ..., X_n$ that satisfy some conditions (so that the covariances exist and so forth) then to me, PCA means forming the matrix $C = (\text{Cov}(X_i, X_j))_{i,j=1,...,n}$. This is a symmetric, positive semidefinite matrix so it can be diagonalized using an orthogonal matrix. I.e. we can find an invertible matrix $A \in \mathbb{R}^{n \times n}$ such that $A^T = A^{-1}$ and 
$$A^{-1} C A = D = \text{diag}(\lambda_1, ..., \lambda_n)$$
One can show that, since $C$ is the covariance matrix of the vector of random variables $X = (X_1, ..., X_n)$ that then the covariance matrix of $AX$ is $A^TCA = D$. This is simple algebra using the mere definition and the bilinearity of covariance (maybe I got it wrong here and one needs to put $A^T$ in front of $X$ instead of $A$ or so). This means that we get new random variables $Y_1, ..., Y_n$ that are linear combinations of the old ones and such that
  $$\text{Cov}(Y_i, Y_j) = 0$$
for all $i \neq j$. However, for $i=j$, i.e. for the diagonal entries we have
  $$\lambda_i = D_{ii} = \text{Cov}(Y_i, Y_i) = \text{Var}(Y_i)$$
i.e. the $i$-th eigenvalue belonging to the $i$-th eigenvector is the variance of the $i$-th resulting PCA component (i.e. the new variable $Y_i$).
What people now do is take a threshold and throw away all $Y_i$ such that $\lambda_i$ is below that threshold. I.e. those variables who having a low variance 'do not contribute much'. However, this is just a heuristic (who sais that just because $Y_i$ has a small variance, it does not contribute much? It may be that this difference is small but unbelievably important ...) and should be used only when necessary, i.e. when you just have 30 or 40 variables and 1 mio. examples and you are using an algorithm that is able to detect uneccessary features then I would not use PCA as this tends to confuse the algorithm...
A: Answer to the edited version of the question:
We must assume here that $Y_1$ is the original data and $Z_1, ..., Z_n$ are the PCA-variables. Then by the simple rule 
$$\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y) + 2ab\text{Cov}(X,Y)$$
(Exercise: prove this rule ;-)) we have
$$\text{Var}(Y_1) = e_{11}^2\text{Var}(Z_1) + ... + e_{1p}^2\text{Var}(Z_p) + 2\sum_{i \neq j} e_{1i} e_{1j} \underbrace{\text{Cov}(Z_i, Z_j)}_{=0}$$ 
So all in all
$$\text{Var}(Y_1) = e_{11}^2\lambda_1 + ... + e_{1p}^2\lambda_p$$
i.e. the variance of the original data component is built up from the entries in the transformation matrix and all the eigenvalues...
Does that go into the right direction?
