In case of semi-supervised data, and PCA pre-processing, should I pre-process all the data or only the labeled data?

Question

suppose I have labeled data and unlabeled data, should I do a PCA process on all the data, and then feed the labeled data through a classifier? Or take only the labeled data through the PCA process and then to the classifier?

EDIT 1:

Adding t-SNE plot of my data (blue and red are labeled and green is unlabeled)

EDIT 2:

1. Goal is to Binary classify the green data (which is unknown) to red or blue (blue is 1, red is 0)

2. t-SNE helps me by giving a picture of how my data spread on a 2D dimension.

3. I hypothesis PCA will help me reduce/compress my data and help me get a better representation of the data (as I only have 4K rows of data and originally, 51 features).

4. Having labeled and un-labeled data doesn't prevent me from solving the problem, but I want to take every advantage I can in order to get a better classification.

Answer 1

If the labeled and unlabeled data are from the same distribution (i.e. generated by the same process) then you should use both for training PCA, as you'll get a better estimate of the covariance matrix. But, if using PCA for dimensionality reduction, keep in mind that it's unsupervised. So, the directions of greatest variance in the input may or may not coincide with the most informative directions for your classification problem.