First of all, I would like to note that I have read similar topics in CrossValidated but I am not fully satisfied.

I have a dataset which consists of an $N\times M$ binary matrix. 1 means that an action is performed and 0 that it is not.

I apply PCA to the dataset and surprisingly get very good results, especially when I reduce it to only two dimensions. I am looking for the intuition behind performing PCA on such a dataset (i.e. where each attribute contains categorical data; you can give whatever example you think is more understandable) and whether a more appropriate technique can be applied. I am working with MATLAB and I need the data in a clustering friendly form.

  • 3
    $\begingroup$ Strictly speaking, one cannot apply PCA to categorical data until they have been numerically coded. You will get different results if you encode your binary values differently. But once the values are encoded, you have--a numeric dataset. Period. There is nothing in any of the accounts of PCA (that I have ever read) that asserts those numeric data must be of a certain type or have certain distributions. PCA is a pithy summary of the locations of those numbers when they are considered to be points in a Euclidean space. From this point of view, there's nothing different about your data. $\endgroup$ – whuber Jul 19 '13 at 17:56
  • 1
    $\begingroup$ @JustCurious, I personally find no sin in doing PCA on binary data, - if you wish to know my opinion. What do you mean saying I need the data in a clustering friendly form, how is this connected with PCA? $\endgroup$ – ttnphns Jul 19 '13 at 18:31
  • 1
    $\begingroup$ With binary data (1=present 0=absent) one might want to set the origin of components at "all variables = absent" point. This will be PCA of the matrix of cosines, rather than of the matrix of correlations. $\endgroup$ – ttnphns Jul 19 '13 at 18:43
  • $\begingroup$ @ttnphns PCA reduces the dimensions and change the values into something that e.g Euclidean distance can be applied on. So, clustering techniques can be applies when data are transformed using PCA. $\endgroup$ – JustCurious Jul 20 '13 at 12:58
  • $\begingroup$ @ttnphns I have read your response to the other question but I am still not convinced or I am a bit confused if you like. Are there any other techniques or PCA adjusted for binary data? $\endgroup$ – JustCurious Jul 20 '13 at 13:01