I am working with autoencoders and have few confusions, I am trying different autoencoders like:
- fully_connected autoencoder
- convolutional autoencoder
- denoising autoencoder
I have two datasets, one is numerical with float and int values, second is a text dataset with text and date values.
Numerical dataset looks like this:
date , id , check_in , check_out , coke_per , permanent_values , temp
13/9/2017 142453390001 134.2 43.1 13 87 21
14/9/2017 142453390005 132.2 46.1 19 32 41
15/9/2017 142453390002 120.2 42.1 33 99 54
16/9/2017 142453390004 100.2 41.1 17 39
Any my text dataset looks like this:
data text
13/9/2017 i totally understand this conversation about farmer market and the organic products, a nice conversation ’cause prices are cheaper than traditional
14/9/2017 The conversation was really great. But I think I need much more practice. I need to improve my listening a lot. Now I’m very worried because I thought that I’d understand more. Although, I understood but I had to repeat and repeat. See you!!!
My questions are:
Should I normalize my numerical data values before feeding to any type of autoencoder? If they are int and float values do I still have to normalize?
Which activation function should I use in autoencoder? Some article and research paper says "sigmoid" and some says "relu"?
Should I use dropout in each layer? Like if my artichare for autoencoder looks like
encoder (1000 --> 500 -- > 256 ----> 128 ) --> decoder (128 --> 256 --> 500--> 784)
something like this?
encoder(dropout(1000,500) --> dropout( 500,256) --> dropout (256,128) )----> decoder(dropout(128,256),dropout(256,500),dropout(500,784))
For text dataset, if I am using
word2vec
or any embedding to convert text into vector then I would have float values for each word, should I normalize that data too?text ( Hello How are you ) -- > word2vec(text) ----> ([1854.92002 , 54112.89774 ,5432.9923 ,5323.98393])
Should I normalize these values or directly use them in autoencoder ?