0
$\begingroup$

Suppose I have a big (1,000x20,000) sparse (95% of elements are zeros) matrix with counts. I want to use autoencoder to encode-decode this matrix. How should I do it? Are there any tricks or guidelines to do it efficiently?

Thank you

$\endgroup$
0
$\begingroup$

I never used autoencoders for sparse data, but my first reaction was "why should this matter?". I found the question interesting, so I made a small Google search and among the first search results, I found an answer by Ian Googfellow to a similar question, who says

You shouldn't need to do anything special for this. Standard good practices for initialization and training should take care of it.

I guess, this could be considered as an authoritative answer for your question.

I don't know how other frameworks, but Keras and TensorFlow support sparse matrices (if it is a matter of memory performance). All you need is a dense, convolutional, or recurrent (depending on nature of your data) layer, or layers, as encoder, and same things for decoder, where on the output layer you would need something like exp function to transform the outputs to non-negative values, if it is counts data (think of Poisson regression).

$\endgroup$
1
$\begingroup$

(Apologize in advance for quite late response)

To my knowledge, for very sparse data you may want to first try out Truncated Single Value Decomposition (SVD), which is implemented in scikit-learn python library. See sklearn documentation / user guide for detail.

There seems to be some research in using Autoencoders for sparse data. This paper proposes a seemingly simple, python-implemented algorithm, and shows it is superior to existing methods including SVD.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.