# Principal components analysis

I was working on an exercise that asks me to get a k-dimensional approximation of a data vector using PCA (containing 4096 dimensions - the faces data set). I did this in R, but the last section asks for a "short mathematical expression for how you construct the approximation where you need to account appropriately for the mean and standard deviation terms". I am familiar how to derive the PC's of the covariance matrix using eigen decomposition but I don't get the "accounting for the mean and sd" part.

Can someone help?

• I think they are asking you to standardize the data before carrying out the PCA. – Vishaal Sudarsan Mar 12 '18 at 1:26

In PCA, our goal is to find linear combinations of the data such that the variance is maximized subject to certain conditions. If our original data is not all on the same scale, and especially if the scale difference is large, then the result can be somewhat meaningless. We can rectify this by standardizing our data first and then running the procedure - this is equivalent to using our original correlation matrix as the covariance matrix in the PCA. This can be shown as follows: $$Cov\left(\frac{X_1-\mu_1}{\sigma_1}, \frac{X_2-\mu_2}{\sigma_2}\right) = \frac{Cov(X_1,X_2)}{\sigma_1\sigma_2} = Corr(X_1, X_2)$$