2
$\begingroup$

I have a very high-dimensional dataset with 27k features. I want to a reduce the dimensionality of the dataset. I want to use PCA to reduce the dimensionality to 2 as the toolbox I am using expects 2 dimensions.

But When I am trying to apply PCA in Matlab I am getting an out of memory error. Can I apply PCA twice to get down to 2 dimesions, i.e. 27k -> intermediate -> 2 dimensions?

Improve this question
$\endgroup$
6
  • $\begingroup$ In theory you could, but it really doesn't make any sense to do so. If you take a pca on your pca, you ignore the fact that some PCs matter more than others. If you really want just two dimensions, why don't you just take the first two PCs (although not knowing your system, this may not make sense)? $\endgroup$
    – Jautis
    Feb 4, 2016 at 14:43
  • $\begingroup$ You mean to say run pca once and the then consider first 2 dimensions of the resultant matrix. ? But I am worried will it still retain the the original information of the original matrix .? $\endgroup$
    – vinaykva
    Feb 4, 2016 at 14:46
  • 4
    $\begingroup$ Please provide the exact commands you are using in Matlab that result in the "out of memory" error. What commands do you use to perform your "27k->intermediate" step? Why doesn't it yield the same error too? $\endgroup$
    – amoeba
    Feb 4, 2016 at 15:05
  • $\begingroup$ yes, it should give the same error for the "27k -> intermediate" part too, so i don't see the point for an intermediate step. however, there is an 'econ' option of matlab's pca, that could be useful. $\endgroup$
    – jeff
    Feb 4, 2016 at 15:11
  • 1
    $\begingroup$ Initially I used to use the below command in matlab A = load(dataset); [coeff,score,latent] = pca(A); It is working fine but it is not giving me 2 dimensions. But I have a tool box from MIT where they use different types of preprocessing the datasets. classification algorithm which I want to implement in that tool box accepts only two dimensions. In that toolbox they provided different preprocessing option to reduce it to 2 dimension. When I used PCA option in their tool box in matlab command it is througing me out of memory error. $\endgroup$
    – vinaykva
    Feb 4, 2016 at 16:35

1 Answer 1

Reset to default
2
$\begingroup$

Have you looked in your dataset if there is constant feature and R^2^ of all the couples of variables. 27K looks a lot.

Otherwise, doing a pca is equivalent to finding a set of linear combination of your features which are orthogonal. Ordered by the dimension explaining the most variance of the original dataset. So you still could do a few PCA on a disjoint subset of your features. If you take only the most important PC, it will make you a new dataset on wish you could do a pca anew. (If you don't, there is no dimension reduction).

But the result will be different from the result given when applying a pca on the full dataset.

Some information will be lost when the most important PC will be taken. And this information could be unbalanced across the subset of original features.

So if you take the main two PC of the final PCA, the result will not the be the two dimension explaining the most variance of the dataset, but the two most distinct dimension of a subset of the dataset.

In fine, the interpretation could be done the same way that a classic pca, but you will have to code all the decomposition of the PC steps.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.