0
$\begingroup$

I have a dataset with about 2 million vectors, the dimension is 200 (D = 200). I want to plot just a few (N = 20) of them in a 2D space. For another, much smaller dataset with a dimension of 20 I did a PCA transformation and plotted the transformed vectors in 2D space:

from matplotlib.mlab import PCA
import matplotlib.pyplot
data = numpy.array( [.. my vectors ..] )
pca = PCA(data)
res = pca.Y
matplotlib.pyplot.scatter(res[:,0], res[:,1])

But PCA just works when the number of vectors is bigger than the dimensions of the vectors: N > D. So, I'm kind of lost now.

Could it be a solution to add D - N + 1 random vectors to my input data, do the PCA transformation and then just plot the N vectors I'm interested in? Or would it be better to use every input vector multiple times (in my example 11 times) to do the PCA transformation?

Is PCA in such a case a viable solution or should I use another MDS method?

Update

It seems my question wasn't as clear as I hoped it would be. I'll try to make things more obvious.

I've got a large array with 2 million vectors whose dimension is 200, but I just want to plot some of them. So, this is the situation:

  • I've got 2,000,000 vectors: M = 2,000,000
  • Dimension of my vectors: D = 200
  • Number of vectors I want to plot: N = 20

Just using those N vectors to do the PCA transformation does not work as the matplotlib.mlab.PCA implementation gives an error if N < D which is the case here.

$\endgroup$
  • 2
    $\begingroup$ PCA just works when the number of vectors is bigger than the dimensions Not true statement. (Or please define what you mean by "works"). $\endgroup$ – ttnphns Jan 30 '17 at 13:49
  • $\begingroup$ Oh really? Matplotlib's PCA yields an error we assume data in a is organized with numrows>numcols and I came across several comments on Stackoverflow that suggested that this is a property of PCA and not just of the matplotlib implementation. So, it's just a matter of using another PCA implementation? $\endgroup$ – z80crew Jan 30 '17 at 14:08
  • $\begingroup$ @amoeba I'm referring to matplotlib.mlab.PCA, which is documented here. As it's documented on the matplotlib homepage under "The Matplolib API" I assumed it would be safe to call it "Matplotlib's PCA". $\endgroup$ – z80crew Jan 30 '17 at 16:11
  • 1
    $\begingroup$ OK, thanks, your question is clearer now, but I still do not understand why you can't do PCA on all your MxD data, and then plot only N vectors. $\endgroup$ – amoeba says Reinstate Monica Jan 30 '17 at 19:44
  • 1
    $\begingroup$ @amoeba Thanks for improving the title. I think the very convenient api of the matplotlib PCA implementation lead me into a wrong direction. It not only calculates the new coordinate system, but projects all the input vectors accordingly. But as I did not need all (2,000,000) input vectors projected, I just used N vectors I wanted to plot as input vectors. Now I've learned that the correct way would be to do the PCA transformation based on as much input as my computer can handle and then project the vectors I want to plot according to the new coordinate system. $\endgroup$ – z80crew Jan 31 '17 at 9:13
1
$\begingroup$

Here is a good discussion of this problem.

PCA when the dimensionality is greater than the number of samples

I would not add a bunch of random vectors to the data set, that will add a lot of noise, and your resulting PCA will not be the real principal components. If you have 2 million vectors, why not just do PCA on a subset (much larger than 20, larger that 200 as well) and plot the PCA transformation of the 20 original data points you want to plot?

$\endgroup$
  • $\begingroup$ Great! I used sample = numpy.array( [data[i] for i in random.sample(range(len(data)), 10000)] ) to generate a subset of 10,000 vectors. Then pca = PCA(sample) to do the PCA transformation. And at last [pca.project(d) for d in data] to project the original vectors according to the transformed coordinates. $\endgroup$ – z80crew Jan 30 '17 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.