0
$\begingroup$

I am new to PCA and have a question about visualization when it comes to fitting and transposing. I have two data, which are train and test. Here are four methods:

# Method 1)
pca = myPCA(n_components = 5) # Conduct myPCA with 5 principal components.
pca.fit(X_train) # Calculate 5 principal components on the training dataset
X_train_pca = pca.transform(X_train)

# Method 2)
pca = myPCA(n_components = 5) # Conduct myPCA with 5 principal components.
pca.fit(X_test)
X_test_pca = pca.transform(X_test)

# Method 3)
pca = myPCA(n_components = 5) # Conduct myPCA with 5 principal components.
pca.fit(X_train)
X_test_pca = pca.transform(X_train)

# Method 4)
pca = myPCA(n_components = 5) # Conduct myPCA with 5 principal components.
pca.fit(X_train)
X_test_pca = pca.transform(X_test)

My question is: Which way is the right way to use PCA for visualization? Method 1)? Method2)? 3)? Or 4)? Although the PCA's tutorial clearly says it needs to be run on the test data, however, it seems that I could not write the right method for this.

Here is my code:

class myPCA():
"""
Principal Component Analysis (A Linear Dimension Reduction Method).
"""

def __init__(self, n_components = 2):
    """
    Conduct myPCA with 2 principal components(the principal and orthogonal modes of variation).
    """
    self.n_c = n_components


def fit(self,X):
    """
    The procedure of computing the covariance matrix.
    """
    cov_mat = np.cov(X.T) # Covariance matrix
    eig_val, eig_vec = np.linalg.eigh(cov_mat) # Eigen-values and orthogonal eigen-vectors in ascending order.
    eig_val = np.flip(eig_val) # Reverse the order, now it is descending.
    eig_vec = np.flip(eig_vec,axis=1) # reverse the order
    self.eig_values = eig_val[:self.n_c] # select the top eigen-vals
    self.principle_components = eig_vec[:,:self.n_c] # select the top eigen-vecs
    self.variance_ratio = self.eig_values/eig_val.sum() # variance explained by each PC

def transform(self,X):
    """
    Compute the score matrix.
    """
    return np.matmul(X-X.mean(axis = 0),self.principle_components) #project the data (centered) on PCs

Visualization Code(still not sure whether to use X_train or X_test below):

import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
figure = plt.figure(dpi=100)
plt.scatter(X_test_pca[:, 0], X_test_pca[:, 1],c=y_test, s=15,edgecolor='none', alpha=0.5,cmap=plt.cm.get_cmap('tab10', 10))
plt.xlabel('component 1')
plt.ylabel('component 2')
plt.colorbar();
$\endgroup$
6
  • 1
    $\begingroup$ Thanks. I still don't really understand your question. What "PCA introduction"? Are you looking at a tutorial somewhere? The title refers to visualizing, but that doesn't show up anywhere in the text. More generally, this is bereft of context; what are you trying to do? $\endgroup$ Mar 17, 2021 at 18:23
  • 1
    $\begingroup$ I don't understand the question, because you appear to be comparing a method to itself: the only things that differ are the (completely irrelevant) names you give to two data subsets in the code. Maybe there are some typographical errors in your initial code blocks? $\endgroup$
    – whuber
    Mar 17, 2021 at 18:23
  • $\begingroup$ FWIW, I'm 90% sure you should do whatever it is you're trying to do on the training set. $\endgroup$ Mar 17, 2021 at 18:24
  • 1
    $\begingroup$ @gung Another way to put it: whatever data you use to fit the PCA is the training set, no matter what you happen to name it in your code! $\endgroup$
    – whuber
    Mar 17, 2021 at 18:24
  • $\begingroup$ Thank you! Thank you all for the help! What about the visualization part? Should I use X_test and y_test instead of X_train and y_train? $\endgroup$
    – mjf88530
    Mar 17, 2021 at 18:30

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.