This is my data,you can see the 1st,2nd,40th and another features has different distribution law.

my data is a matrix, there is one box per column.On each box, the central mark is the median, the edges of the box are the 25th and 75th percentiles, the whiskers extend to the most extreme data points not considered outliers, and outliers are plotted individually.

enter image description here

I want to use this as input and output of an LSTM anto-encoder.

So how to normalize the data?

I try mean-std and find it better than min-max rule(normalize the data to [0,1]) when feeding into LSTM.

min-max rule:

enter image description here

mean-std rule:

enter image description here

But it's still a not very perfect rule to fit this problem.

  • 1
    $\begingroup$ If you normalize the data you may lose what look like very important aspects of it. Are you certain that you want to use the same model for all of this data? $\endgroup$ – David Kozak Nov 23 '17 at 1:08
  • $\begingroup$ No,can we use min-max and mean-std and something else way to deal with these data? $\endgroup$ – partida Nov 23 '17 at 1:14
  • $\begingroup$ You can use sklearn scalers as StandartScaler, fit in one dataset and transform the others, or fit and transform in each one. They have an example doing that. $\endgroup$ – Adelson Dias Nov 23 '17 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.