4
$\begingroup$

I have a $152 \times 27578$ matrix, $152$ samples and $27578$ features, and I used the PCA function for the dimension reduction in Matlab.

X = load(dataset);
coeff = pca(X);

It generated a $27578 \times 151$ matrix. But I don't understand what exactly it is generating and I am unable to understand what to do next. Can any one help me with the understanding? My main goal is to reduce the dimension of my original matrix.

Improve this question
$\endgroup$
4
  • $\begingroup$ Are you sure coeff is a $27578\times 151$ matrix ? Have you tried looking here ? $\endgroup$ – Gilles Jan 27 '16 at 15:28
  • $\begingroup$ @Gilles Ya I am sure it generated 27578 X 151. and I referred the link which you mentioned $\endgroup$ – vinaykva Jan 27 '16 at 19:41
  • $\begingroup$ @Gilles output matrix is [27578 X 151] is it because it is considering the first column as case and control ..?? $\endgroup$ – vinaykva Jan 27 '16 at 20:07
  • $\begingroup$ Never mind my deleted comment. $\endgroup$ – Gilles Jan 27 '16 at 20:52
5
$\begingroup$

If you type help pca you will see loads of information about the function.

If you only output one argument, it will return the principal coefficients, sometimes called the loadings. The $27578\times151$ matrix you received contains the first loading in the first row, the second in the second row and so on.

If you ask for two outputs, you obtain

[V, U] = pca(X);

where V contains the loadings and U the score values. You reconstruct the input data by U*V'.

In order to perform dimensionality reduction, you must select the first n components of both matrices as U(:, 1:n) and V(:, 1:n) and perform the approximated reconstruction as U(:, 1:n)*V(:, 1:n)'.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks for the explanation but how to choose the value of n that gives me the matrix without loosing the information. Can I plot the graph and to select that. If yes how do I do that ..? $\endgroup$ – vinaykva Jan 28 '16 at 0:47
  • $\begingroup$ Aaah, that's the $\$1M$ question! ;-) In fact, there's no one way to do that. There are several different approaches. Some people just look at the singular values, and look for a quick drop in their values. You can e.g. plot(sqrt(sum(U.^2, 1))) and do a visual inspection. You could perform a cross-validation and estimate the left-out rows using the loadings of a PCA on the kept rows. I've seen $\chi^2$ tests be used, and I'm sure there are plenty of other ways. Do a web search for "pca number of components" and I'm sure you will have enought to read for several days ;-) $\endgroup$ – Tommy L Jan 28 '16 at 7:42
  • $\begingroup$ Using all of the principle components will allow you to retain all the original information, however, obviously you will not have reduced the dimensional. The trick of PCA is to choose the minimum number of components that still describes most (say~99%) of the variance in your dataset. I have linked to a method in my answer. $\endgroup$ – CatsLoveJazz Jan 28 '16 at 9:31
  • $\begingroup$ I did tried that to plot the values of U. I am getting a quick drop at around 8 or 9. That's reason I started to get doubt. Is that correct and when I tried to multiply with the value of 8. result_30758 = (U(:,1:8)*V(:,1:8)'); I am getting the original matrix size which is 152 X 27578. Is this correct .? $\endgroup$ – vinaykva Jan 28 '16 at 14:11
  • 1
    $\begingroup$ The rank of your matrix will be 8 instead of 151. If you want to reduce the dimension of your data matrix (meaning the number of variables) then you can work with the U matrix instead of the original data matrix X in your analysis. The vectors of U are just linear combinations of the columns of X. $\endgroup$ – Tommy L Jan 28 '16 at 14:42
2
$\begingroup$

The output of PCA is a matrix of your principle components. So, this matrix contains a set of new signals but now these components are ordered in terms of how much of the datasets variance they capture. The first principle component in the matrix coeff(1,:) describes the most variance to the last component coeff(27578,:) which captures the least. So you simply need to choose $k$ dimensions you want to reduce your new inputs to like so coeff(1:k,:)

One method suggested here would be to select the first $k$ components that still captures 99% of the variance of your dataset. Thus you would have reduced dimension inputs but still describe your data well.

Improve this answer
$\endgroup$
1
  • $\begingroup$ you mean to say I should choose the first K and than Should I multiple that with my original matrix to get the dimension reduction matrix $\endgroup$ – vinaykva Jan 27 '16 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.