Neural network regression with confidence interval implemented with Keras When using neural network for classification problem, and using softmax as last layer for last layer.
Typically, we have a prediction and a confidence level. However, is there such confidence interval measure for neural network regression problem? 
 A: You would have to output vectors of means and standard deviations rather than discrete values to achieve that.
One solution to get those vectors would be variational inference - generate those, sample w/reparametrization, then optimize so the results of the sampling match the original values like in normal regression (i.e. MSE/MAPE/MAE/whatever loss) and regularize the means and stddev to 0/1 respectively.
Essentially the same process as a vanilla Variational Autoencoder, except you're not bound by the Autoencoder architecture, and you want the means/stddevs as the outputs of the trained network rather than the sampled values.
A: Thanks to @jkm I was quite fascinated by the idea of implementing confidence intervals for regression in Keras. Personally, I hope it would boost my model network's performance as I have multiple supervised DL models on different datasets that are being fed into a single Reinforcement Learning Algorithm. Hence, the RL 'parent' would be able to know how confident the individual model is at a given time.
I went with a different approach to implementing a custom loss function (as I need the two neuron outputs without any Keras magic of printing hidden layer outputs...). If at the end of the model you have a Dense(2) layer, then you have an output of size [batch_size, 2]. The problem is that tf.random.normal needs two separate tensors for the mean and std. So you need to transpose the output.
The Loss Function with MSE:
class RegressionDistLoss(tf.keras.losses.Loss):
    def __init__(self):
        super(RegressionDistLoss, self).__init__()
        self.mse = tf.keras.losses.MeanSquaredError()

    def call(self, y_true, y_pred):
        x = tf.transpose(y_pred)
        x = tf.random.normal([y_pred.shape[0]], x[0], x[1])
        # print(x) # tf.Tensor([-0.2107901  3.8580756  1.8032494], shape=(3,), dtype=float32)
        return self.mse(y_true, x)

Val. Experiment:
y_true = tf.constant([0, 4, 2], dtype='float32')
y_pred = tf.constant([[0, 1], [3.5, .5], [2.2, .7]], dtype='float32')
x = RegressionDistLoss()(y_true, y_pred)
print(x) # <tf.Tensor: shape=(), dtype=float32, numpy=0.0344286>

