Restrict output range of a neuron based on output of other neurons

I have a neural network with three output neurons $$X_1$$, $$X_2$$ and $$X_3$$ with output range in [-1, 1].

I have many training data split in 80:20 ratio between training & testing sets. While looking at the output of the neural network, I noticed a big thing that the neural network haven't learned.

There is a restriction in the output as $$X_1 < X_2 < X_3$$.

I found that only with training, I can't achieve this restriction. Is there a recommended way theoretically or using Keras library to tell the neural network about this restriction.

You can enforce the constraint with an additional transformation. Suppose we have $$x_i = \sigma(AH + b)$$ where $$\sigma$$ is the sigmoid activation and $$A, b$$ are the weights and biases.

We can introduce a fixed matrix $$W = \begin{bmatrix}1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 1 & 1\end{bmatrix}$$, which is not updated during back-propagation (that is, it is non-trainable). We know that $$Wx$$ will satisfy the constraint because $$x_i$$ are all positive, so we have $$0 < x_1 < x_1 + x_2 < x_1 + x_2 + x_3 < 3$$.

If you need the result to be in $$[-1,1]$$, then you can just rescale and shift: $$\frac{2}{3}Wx - 1$$. The choice of $$\frac{2}{3}$$ means that the largest value possible is 2, and the smallest possible value remains 0. Shifting by $$-1$$ moves the interval from $$[0,2]$$ to $$[-1,1]$$.