Formatting input data to Scikit learn for Kmean and PCA

Question

I am very confused about the data that feeds to Kmean and PCA algorithm using Scikit Learn command in Python. I searched a lot in the internet but no where I found the clear answer.

I have $X$, a $m \times n$ matrix where $m$ is the number of samples and $n$ is number of features. For example consider the matrix to be $20 \times 10$ where we have $20$ different people with $10$ different features like height, weight and so on.

kmeans = KMeans(n_clusters=2,init='random')
kmeans.fit(X)

I am using above command for the Kmean and following command for the PCA :

pca = PCA(n_components=2,svd_solver='full')
Xbar = pca.fit_transform(X)

Therefore I am feeding matrix X directly to the Kmeans and PCA (assume I already standardized the input). For the Kmeans I am not getting anything strange but when I print the Xbar as two principle components I have vector of 20*1 for each eigenvector. Shouldn't I get 2 eigenvectors of $10\times 1$ since the covariance matrix should be $10\times10$?

I would like to know is this the right of way expressing the input data meaning that should the rows of matrix be number of samples and columns be number of features?

Answer 1

The scikit-learn standard is to pass a design matrix $X \in \mathbb{R}^{n\times d}$ such that $n$ is the number of samples and $d$ the number of features, so you are passing the data in the right format.

From your question, I presume you expect fit_transform( ) method to return the eigenvectors of the covariance matrix with the highest associated eigenvalue (i.e., the PCA projection matrix). However, fit_transform() instead returns the projected data; this is, the data matrix you passed multiplied by the projection matrix. Hence the output you get is $n\times 2$, as it represents your $n$ samples projected onto a two-dimensional output space.

If you want to get the actual projection matrix, you can get that by accessing the components_ attribute of the PCA object (after you fit it, of course):

components_ : array, shape (n_components, n_features)