# Doing PCA with $m$ vectors in $d$ dimensions and then plotting only $n$ vectors, when $n<d<m$

I have a dataset with about 2 million vectors, the dimension is 200 (D = 200). I want to plot just a few (N = 20) of them in a 2D space. For another, much smaller dataset with a dimension of 20 I did a PCA transformation and plotted the transformed vectors in 2D space:

from matplotlib.mlab import PCA
import matplotlib.pyplot
data = numpy.array( [.. my vectors ..] )
pca = PCA(data)
res = pca.Y
matplotlib.pyplot.scatter(res[:,0], res[:,1])


But PCA just works when the number of vectors is bigger than the dimensions of the vectors: N > D. So, I'm kind of lost now.

Could it be a solution to add D - N + 1 random vectors to my input data, do the PCA transformation and then just plot the N vectors I'm interested in? Or would it be better to use every input vector multiple times (in my example 11 times) to do the PCA transformation?

Is PCA in such a case a viable solution or should I use another MDS method?

Update

It seems my question wasn't as clear as I hoped it would be. I'll try to make things more obvious.

I've got a large array with 2 million vectors whose dimension is 200, but I just want to plot some of them. So, this is the situation:

• I've got 2,000,000 vectors: M = 2,000,000
• Dimension of my vectors: D = 200
• Number of vectors I want to plot: N = 20

Just using those N vectors to do the PCA transformation does not work as the matplotlib.mlab.PCA implementation gives an error if N < D which is the case here.

• PCA just works when the number of vectors is bigger than the dimensions Not true statement. (Or please define what you mean by "works"). Jan 30 '17 at 13:49
• Oh really? Matplotlib's PCA yields an error we assume data in a is organized with numrows>numcols and I came across several comments on Stackoverflow that suggested that this is a property of PCA and not just of the matplotlib implementation. So, it's just a matter of using another PCA implementation? Jan 30 '17 at 14:08
• @amoeba I'm referring to matplotlib.mlab.PCA, which is documented here. As it's documented on the matplotlib homepage under "The Matplolib API" I assumed it would be safe to call it "Matplotlib's PCA". Jan 30 '17 at 16:11
• OK, thanks, your question is clearer now, but I still do not understand why you can't do PCA on all your MxD data, and then plot only N vectors. Jan 30 '17 at 19:44
• @amoeba Thanks for improving the title. I think the very convenient api of the matplotlib PCA implementation lead me into a wrong direction. It not only calculates the new coordinate system, but projects all the input vectors accordingly. But as I did not need all (2,000,000) input vectors projected, I just used N vectors I wanted to plot as input vectors. Now I've learned that the correct way would be to do the PCA transformation based on as much input as my computer can handle and then project the vectors I want to plot according to the new coordinate system. Jan 31 '17 at 9:13

• Great! I used sample = numpy.array( [data[i] for i in random.sample(range(len(data)), 10000)] ) to generate a subset of 10,000 vectors. Then pca = PCA(sample) to do the PCA transformation. And at last [pca.project(d) for d in data] to project the original vectors according to the transformed coordinates. Jan 30 '17 at 17:38