I am working on implementing PCA on the MNIST dataset and have calculated the eigen vector and eig Values from the co-variance matrix. Now I want to have a new co-ordinate system represented by PC1 and PC2 just so I can project the datapoints on PC1. I have a code snippet where the new coordinate is created by taking the dot product of the transpose of eigen vectors with the transpose of the standardized data matrix.

eigValues, eigVectors = linalg.eigh(Covariance_mat, eigvals= (782, 783))
new_cordinates = np.matmul(eigVectors.T, standardize_mnist.T)

array([[-5.2264454 ,  6.03299601, -1.70581328, ...,  7.07627667,
    -4.34451279,  1.55912058],
   [-5.14047772, 19.29233234, -7.64450341, ...,  0.49539137,
     2.30724011, -4.80767022]])

Now, I am not sure about the intuition behind this dot product, which leads to a new 2d coordinate system.

I am very new to Machine Learning hence, could someone please help me understand this part.


1 Answer 1


Let the dimension be denoted by $d$. There are $d$ eigenvectors of dimension $d\times1$. In PCA< we choose $k<d$ of them an obtain an underrepresentation. Typically, the data matrix, $X$, as also in your case has dimension $n\times d$, where $n$ is number of data points in the dataset. So, each row is a sample. In order to convert a sample, say $x$ of dimension $1\times d$, to new coordinate system, we need project the sample onto the chosen set of eigenvectors, let them be denoted by $v_1,v_2...v_k$. Projection means calculating dot product, and projecting $x$ onto each eigenvector is equivalent to calculating the dot product $v_i^Tx^T$. So, the new coordinates for our sample will be the following: $$x_k=[v_1^Tx^T,\ v_2^Tx^T\dots v_k^Tx^T]=[v_1^T\dots v_k^T]x^T=V^Tx^T$$ If you do this for all the samples in the dataset, we have: $$X_k=V^T\begin{bmatrix}x_1^T\\x_2^T\\\vdots\\x_n^T\end{bmatrix}=V^TX^T$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.