I am working on a regression problem to predict 3 outputs from 5 inputs, The inputs range from -30 to 30 except for one input that ranges from 20000 to -2e7. The 3 outputs range from 0 to 2e6, I am using Keras API and my network is simple 3 hidden layers (32169),
I am using leaky relu and Adam optimizer and training over 500 epochs with a batch size= 64. I use sklearn standardscaler() for standardizing the data.
My problem is that the network doesn't learn and the prediction I am getting are not accurate at all!! I tried complicating the network by adding layers and units but it doesn't work at all, I also tried using different normalization methods like minmax() and tanh estimator but no improvements were noticed!!
I tried many combinations of learning rates (0.1 to 0.000001) also epochs=(100 to 1000000), I tried changing batch size (10 to 256) no luck at all.
I tried different activation functions (relu,elu...etc) also tried different optimizers(RMSprop, SGD, adagrad, adam ...etc) no improvement at all!!!
My validation loss typically goes from around 1 to 0.3 and stops improving, I tried dividing the network into 3 networks where each predicts only one output but it didn't improve anything!!
and this is my learning curve:
These are the output data distributions
and here are the input data distributions:
There is no relation between the inputs and the outputs!! Can anyone help me with this problem?! thank you!