0
$\begingroup$

I am trying to reconstruct the acceleration values of a tri-axial accelerometer using a simple autoencoder. As acceleration values are often negative (e.g, -3.4) therefore using a ReLU activation function clips the negative values to zero and a sigmoid function gives output only in interval 0 to 1.

Here is a diagram showing the autoencoder which as only 2 hidden layers and takes an input of size (1x3) (x,y, z-axis) and tries to reconstruct same values at the output layer (x^,y^,z^)

enter image description here

what would be a good solution for this problem?

$\endgroup$
0
$\begingroup$

TanH gives output from (-1; 1).

ReLu is cut off negative values same as sigmoid, however weights could be negative too so you can expect training process will leads to get negative weights to handle this problem. Second thing is that an input and an output to neural network should be translated into domain [-1;1] or [0;1] so if you choose second option problem of negative values disappear. If you choose [-1;1] NN could easily handle this by biases.

However if you think that using ReLu seems to be unnatural for this problem. You can also try using different activation function, i.e choose sigmoid on hidden layers and pre-train them separetly to speed-up handling with vanishing gradient.

$\endgroup$
  • $\begingroup$ Thanks for your answer. I tried normalizing acceleration values ([-1,1]) and applied tanh activation. But normalizing makes some values so small that results are even worse. $\endgroup$ – Salman Shaukat Jan 30 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.