Doing PCA with $m$ vectors in $d$ dimensions and then plotting only $n$ vectors, when $n<d<m$

Question

I have a dataset with about 2 million vectors, the dimension is 200 (D = 200). I want to plot just a few (N = 20) of them in a 2D space. For another, much smaller dataset with a dimension of 20 I did a PCA transformation and plotted the transformed vectors in 2D space:

from matplotlib.mlab import PCA
import matplotlib.pyplot
data = numpy.array( [.. my vectors ..] )
pca = PCA(data)
res = pca.Y
matplotlib.pyplot.scatter(res[:,0], res[:,1])

But PCA just works when the number of vectors is bigger than the dimensions of the vectors: N > D. So, I'm kind of lost now.

Could it be a solution to add D - N + 1 random vectors to my input data, do the PCA transformation and then just plot the N vectors I'm interested in? Or would it be better to use every input vector multiple times (in my example 11 times) to do the PCA transformation?

Is PCA in such a case a viable solution or should I use another MDS method?

Update

It seems my question wasn't as clear as I hoped it would be. I'll try to make things more obvious.

I've got a large array with 2 million vectors whose dimension is 200, but I just want to plot some of them. So, this is the situation:

• I've got 2,000,000 vectors: M = 2,000,000
• Dimension of my vectors: D = 200
• Number of vectors I want to plot: N = 20

Just using those N vectors to do the PCA transformation does not work as the matplotlib.mlab.PCA implementation gives an error if N < D which is the case here.

Answer 1

Here is a good discussion of this problem.

PCA when the dimensionality is greater than the number of samples

I would not add a bunch of random vectors to the data set, that will add a lot of noise, and your resulting PCA will not be the real principal components. If you have 2 million vectors, why not just do PCA on a subset (much larger than 20, larger that 200 as well) and plot the PCA transformation of the 20 original data points you want to plot?