# Why do we get a new co-ordinate system when we dot product the transpose of eigen vectors with the transpose of a matrix

I am working on implementing PCA on the MNIST dataset and have calculated the eigen vector and eig Values from the co-variance matrix. Now I want to have a new co-ordinate system represented by PC1 and PC2 just so I can project the datapoints on PC1. I have a code snippet where the new coordinate is created by taking the dot product of the transpose of eigen vectors with the transpose of the standardized data matrix.

eigValues, eigVectors = linalg.eigh(Covariance_mat, eigvals= (782, 783))
new_cordinates = np.matmul(eigVectors.T, standardize_mnist.T)
new_cordinates

Output:

array([[-5.2264454 ,  6.03299601, -1.70581328, ...,  7.07627667,
-4.34451279,  1.55912058],
[-5.14047772, 19.29233234, -7.64450341, ...,  0.49539137,
2.30724011, -4.80767022]])


Now, I am not sure about the intuition behind this dot product, which leads to a new 2d coordinate system.

Let the dimension be denoted by $$d$$. There are $$d$$ eigenvectors of dimension $$d\times1$$. In PCA< we choose $$k of them an obtain an underrepresentation. Typically, the data matrix, $$X$$, as also in your case has dimension $$n\times d$$, where $$n$$ is number of data points in the dataset. So, each row is a sample. In order to convert a sample, say $$x$$ of dimension $$1\times d$$, to new coordinate system, we need project the sample onto the chosen set of eigenvectors, let them be denoted by $$v_1,v_2...v_k$$. Projection means calculating dot product, and projecting $$x$$ onto each eigenvector is equivalent to calculating the dot product $$v_i^Tx^T$$. So, the new coordinates for our sample will be the following: $$x_k=[v_1^Tx^T,\ v_2^Tx^T\dots v_k^Tx^T]=[v_1^T\dots v_k^T]x^T=V^Tx^T$$ If you do this for all the samples in the dataset, we have: $$X_k=V^T\begin{bmatrix}x_1^T\\x_2^T\\\vdots\\x_n^T\end{bmatrix}=V^TX^T$$