Derive vectors from sklearn.decomposition PCA (python)

I have a 3d scatter plot. After performing a PCA, I was to find a way to plot eigenvectors in the original feature space. I saw an example of how to do it in 2d, but I cannot seem to translate it well to a 3d via matplotlib.pyplot nor do I understand how to breakdown the attributes returned via the library (here this community I guess could help).

the simple part of the scatter plot is below. If I could get an explanation of how to use any of the following attributes to get what I need, it would be great: components_,explained_variance_, explained_variance_ratio_, singular_values_, mean_ (I understand what they mean, but not their role in the big picture of constructing a vector)

EDIT: The main thing I need (for the next steps) is the vectors themselves and the plane the first 2 vectors form.

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(x, y, z)
plt.show()


Ok I try my best to explain some of terms, the eigenvectors are stored under pca.components_. In the post you linked, they took the explained_variance_ to scale the length of the eigenvector by its eigenvalue, but for plotting you most likely don't need that. So I use the iris data:

import pandas as pd
import seaborn as sns
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
import numpy as np

pca = PCA(n_components=2)
pca.fit(df)

df = df.loc[df['species']!="setosa"][['sepal_length','sepal_width','petal_length']]


And set up the plot using the code you have:

fig = plt.figure(figsize=(12, 12))
ax = fig.add_subplot(111, projection='3d')
ax.scatter(df['sepal_length'],df['sepal_width'], df['petal_length'])
cols = ['r','k']
for i in range(len(pca.components_)):
PC = list(zip(pca.mean_-2*pca.components_[i],pca.mean_+1*pca.components_[i]))
ax.plot(PC[0],PC[1],PC[2],cols[i])
plt.show()


Going through the part of the code:

PC = list(zip(pca.mean_-2*pca.components_[i],pca.mean_+1*pca.components_[i]))


Your PCA is always performed on the centered matrix, so we need find the centre of the data, which is stored under pca.mean_ . Now, it remains to define the starting and end point of the line that passes through this centre like below:

And for simplicity sake i use, +/- 1 eigenvector, and you zip part is to put all the x, y and z coordinates together, for use in ax.plot Then what remains is to iterate through the components.

If you would like to scale the eigenvectors by the eigenvalue, you just need to replace the 1.

This is also a nice blog that explains it, together with post such as this and this

• thank you, it is very helpful. However, I have a couple of question: what does the Col variable serve as? EDIT (had another question but I understood the answer to it) I have tried to understand these last couple of lines in order to both add the 3d PC which, if I'm not mistaken should be the normal of the plane formed by the first 2 PCs, and to get an equation of these vectors in order to calculate an angle between the 3rd component and another plane in the next stage. May 7, 2020 at 18:45
• pca.mean_ is a numpy array, it's not like a list where you have to iterate over for subtraction. np.array([1,2,3]) - 1 gives you array([0, 1, 2]). is this what you are referring to? so I have the eigenvectors, which is an array of the same length subtracted from pca.mean May 7, 2020 at 19:10
• cols is just to have the colors in a list, so you can iterate of each of the principal components... yes more or less. you also need to change the pca calculation part to give you 3 components. May 7, 2020 at 19:12