I have time sequences of 2D heat maps. For different people I have heat maps over time. For each person I have around 720 heat maps and in total around 50'000 heat maps.

Now, I would like to train an autoencoder on these heat maps to learn a meaningful low dimensional representation. I will use Keras in Python for the implementation.

I thought about using a CNN for encoder and decoder (but I'm unsure about the number of layers and the size of the layers). Another option could be to use a cycleGAN (pix2pix) or also taking time into account using LSTM but I don't know how to use LSTM for 2D image data. I have also heard that attention layers might be interesting.

What architectures and configurations are reasonable to use? And what loss function would you use?

  • $\begingroup$ Without actual details on what you plan on doing with the representations, there's no single valid answer here. Rather than asking which is best, just try one and see if it fits your use case. $\endgroup$ – Alex R. Jul 13 '19 at 17:12
  • $\begingroup$ @AlexR.I would like to use the low dimensional representation of the autoencoder as feature for a SVM classification. $\endgroup$ – machinery Jul 13 '19 at 18:49
  • 1
    $\begingroup$ If you already have class labels, why not just train the model end-to-end, with a CNN followed by a classification layer? $\endgroup$ – Alex R. Jul 13 '19 at 18:53
  • $\begingroup$ @AlexR.That is not possible because I have too few labels (only around 700 per class for 2 classes). $\endgroup$ – machinery Jul 13 '19 at 21:57
  • 1
    $\begingroup$ It doesn't make a lot of sense to use 3 colors, since that's just a visual aid to communicate different levels of heat. Heat can be reduced to a single dimension, like temperature. Are you able to infer this from the raw data? $\endgroup$ – Alex R. Jul 14 '19 at 2:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.