MLP for regression not learning enough? I am working on a regression problem to predict 3 outputs from  5 inputs, The inputs range from -30 to 30 except for one input that ranges from 20000 to -2e7. The 3 outputs range from 0 to 2e6, I am using Keras API and my network is simple 3 hidden layers (32169),
I am using leaky relu and Adam optimizer and training over 500 epochs with a batch size= 64. I use sklearn standardscaler() for standardizing the data.
My problem is that the network doesn't learn and the prediction I am getting are not accurate at all!! I tried complicating the network by adding layers and units but it doesn't work at all, I also tried using different normalization methods like minmax() and tanh estimator but no improvements were noticed!!
I tried many combinations of learning rates (0.1 to 0.000001) also epochs=(100 to 1000000), I tried changing batch size (10 to 256) no luck at all.
I tried different activation functions (relu,elu...etc) also tried different optimizers(RMSprop, SGD, adagrad, adam ...etc) no improvement at all!!!
My validation loss typically goes from around 1 to 0.3 and stops improving, I tried dividing the network into 3 networks where each predicts only one output but it didn't improve anything!!
this is my model:

and this is my learning curve:

These are the output data distributions

and here are the input data distributions:

There is no relation between the inputs and the outputs!!
Can anyone help me with this problem?! thank you!
 A: Your learning curve looks good, mean squared error is decreasing for both test- and training set, and after a while the network has learned what it can apparently. "Not enough" is subjective, but this looks like it's learning, no obvious problems.
That said, I have the following suggestions

*

*Scale your inputs to resemble a normal distribution. For input 1 to 3, the effect will be limited, you should just divide by standard deviation (be sure to estimate it from the test distribution). For the other two, apply a log-scaling first, then scale it.

*Also scale the output variable using a logarithm and scale it

*Your network seems a bit heavy, reduce the number of layers to two, and reduce the number of nodes in the hidden layers as well. For example, use

5 input - 5 hidden layer - 3 output
5 input - 3 hidden layer - 3 output
EDIT: if it is true that inputs and outputs are independent, this whole exercise is pointless. You cannot make good predictions if your inputs don't say anything about the outputs. Good on @Dave for catching that fundamental issue.
EDIT: Then the question is; how can the MSE still decrease? I'm guessing then this is because the network is learning the average outputs. If you initialize a network, especially with such unscaled predictors and outcomes, it will do much worse than predicting the average of the outputs. It will adapt the parameters so that it predicts the average for all outputs, which is the best you can do given independent (in other words, useless) information.
A: You said that the inputs and outputs are independent. In that case, you’re getting the right answer. You should not be able to use the inputs to predict the outputs.
That’s what independence means: knowing something about one gives no insight into the other.
