Autoencoders and/or PCA for highly sparse float vectors and a dataset of more than 2 million examples

I have a highly dimensional sparse dataset composed of 2.5 million of examples as follow :

dataset_dimension=[2500000,360,280,18]

Each example of this dataset is of dimension (360,280,18). Examples are very sparse and real-valued. values are between 0 and 1, we have 0 values, values which are near to 0 such as 0.000000000000000000000001245 and high values such 0.8. At each of the 18 dimensions we have at most one value which is high and the remaining either are very near to 0 or equal 0. Hence for each example we have at most 18 values which near to one and the remaining are very small of 0 values. In average in each example of dimension (360,280) we have at most 500 values which are different from zeros.

What l want to do ? l would like to reduce the dimensionality of my dataset and encode these examples as smaller dense vectors.

What are my ideas ?

1)Kernel PCA. However, it seems that my dataset is so huge to fit into memory in order to ouput the K first components that cover 95% variability of my data. To apply PCA on my dataset, we transform dataset_dimension=[2500000,360,280,18] to dataset_dimension=[2500000,360*280*18] where 360*280*18=1814400. It is a very high dimensional sparse vector. It is caracterised by a spartsity of about 500/1814400.

Is sickit learn package able to apply PCA on a dataset of [2500000,1814400] ?

2) Apply an autoencoder and extract the intermediate representation (low level representation , let's a 2048-vector dense representation). However, since my input are at the same time : - In a very high dimension : 1814400 - with a large sparsity : 500/1814400 - values are either equal 0 or few of them near to 0 and the remaining are near to 0 such as 0.0000000000000001245, 0.0000000000000000000000045. Is autoencoder able to learn a dense representation from this kind of inputs ? Is autoencoder not sensible to high dimensional and large sparse float-valued vectors ?

Thank you for your consideration

• Your small, positive values are awfully close to double-float machine precision. Are you sure that they are not merely the result of accumulated numerical error? – Sycorax Aug 10 '18 at 21:57
• no they are not. I can ignore them and turn them to 0. because for me the most important values are those > 0.1 – vincet Aug 10 '18 at 22:00
• I think you have a slightly issue that you really want sparse PCA rather than kernel PCA. Just doing kernel PCA will still return large dense eigenvectors and scores. That said, for a first step you might want to use something like RSpectra that uses sparse matrices of class dgCMatrix and dgRMatrix. – usεr11852 Aug 11 '18 at 0:27