I have an original data set with a number of features N equal to 135 and a number of rows equal to 32000. The last column of the data set ( column 136 ) can take either -1 or 1 depending on the class of the row. Each of the features can take a binary value of 0 or 1 ( characteristic present or not).
The original data set looks like this:
V1 V1 V3 V4 V5 ............ V136 0 1 0 0 1 -1 0 1 1 0 1 -1 1 0 1 1 0 1 . . . . . . . . . . . . . . . . . . ( row 32000)
I am required to use PCA to reduce the dimentionality of my data, and then apply SVM for classification. when I used PCA to cover 96% of the variance, I got a number of principal componenents equal to 54 ( which is good, since I reduced the number of features from 135 to 54), and then I computed the reduced data matrix which looks like this:
V1 V2 V3 V4 ................. V54 Class 1.89463604908794 -1.15458215287551 -0.577464752091365 -0.560850946997309 -1 -0.170737500644998 0.190545093723431 0.451249831953581 1.44451854658279 -1 ...... ......
When I run SVM on the original data set, SVM finishes quickly, however, when I run it on the reduced data set, it takes forever. A part of the problem is that the original data set was binary, and thus it takes less memory when loaded. However, because the reduced data set is composed of real number, it takes more memory. For comparison purposes, I also saved the reduced data set to disque, and I compared the size of the two files in terms of KB:
Original Data ( not reduced) : 8500 KB Reduced Data set : 32200 KB
It looks like PCA reduced the number of features but it increased the total size in term of KB. Any ideas about how I can overcome this issue? Doesn't this throw away the original purpose of PCA ( reducing the dimentionality, and thus the size)?