Derive vectors from sklearn.decomposition PCA (python)

Question

I have a 3d scatter plot. After performing a PCA, I was to find a way to plot eigenvectors in the original feature space. I saw an example of how to do it in 2d, but I cannot seem to translate it well to a 3d via matplotlib.pyplot nor do I understand how to breakdown the attributes returned via the library (here this community I guess could help).

the simple part of the scatter plot is below. If I could get an explanation of how to use any of the following attributes to get what I need, it would be great: components_,explained_variance_, explained_variance_ratio_, singular_values_, mean_ (I understand what they mean, but not their role in the big picture of constructing a vector)

EDIT: The main thing I need (for the next steps) is the vectors themselves and the plane the first 2 vectors form.

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(x, y, z)
plt.show()

Answer 1

Ok I try my best to explain some of terms, the eigenvectors are stored under pca.components_. In the post you linked, they took the explained_variance_ to scale the length of the eigenvector by its eigenvalue, but for plotting you most likely don't need that. So I use the iris data:

import pandas as pd
import seaborn as sns
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
import numpy as np

pca = PCA(n_components=2)
pca.fit(df)

df = pd.read_csv("https://raw.githubusercontent.com/uiuc-cse/data-fa14/gh-pages/data/iris.csv")
df = df.loc[df['species']!="setosa"][['sepal_length','sepal_width','petal_length']]

And set up the plot using the code you have:

fig = plt.figure(figsize=(12, 12))
ax = fig.add_subplot(111, projection='3d')
ax.scatter(df['sepal_length'],df['sepal_width'], df['petal_length'])
cols = ['r','k']
for i in range(len(pca.components_)):
    PC = list(zip(pca.mean_-2*pca.components_[i],pca.mean_+1*pca.components_[i]))
    ax.plot(PC[0],PC[1],PC[2],cols[i])
plt.show()

Going through the part of the code:

PC = list(zip(pca.mean_-2*pca.components_[i],pca.mean_+1*pca.components_[i]))

Your PCA is always performed on the centered matrix, so we need find the centre of the data, which is stored under pca.mean_ . Now, it remains to define the starting and end point of the line that passes through this centre like below:

And for simplicity sake i use, +/- 1 eigenvector, and you zip part is to put all the x, y and z coordinates together, for use in ax.plot Then what remains is to iterate through the components.

If you would like to scale the eigenvectors by the eigenvalue, you just need to replace the 1.

This is also a nice blog that explains it, together with post such as this and this