# Constrain a Neural Network to be monotonic?

Is there a way or theorem that allows me to constrain a function $N(x)$, where $N(x)$ is a feedforward mlp such that it will always be a monotonic function. The simplest way is of course to use a penalty method, but this does not 100% guarantee, I am looking to see if there is a function or transformation that can be applied to $N(x)$ that constrains its form to being monotonic. The only way I can currently think of is by making all weights $w_{i,j,k} >0$ and that the activation function $f(a)$ are monotonic and non-negative (using $f(a) = \frac{tanh(a)+1}{2}$) and having a zero bias.

Here's an example of an early publication in this vein.

Joseph Sill. "Monotonic Networks". California Institute of Technology. 1998.

Monotonicity is a constraint which arises in many application domains. We present a machine learning model, the monotonic network, for which monotonicity can be enforced exactly, i.e., by virtue of functional form. A straightforward method for implementing and training a monotonic network is described. Monotonic networks are proven to be universal approximators of continuous, differentiable monotonic functions. We apply monotonic networks to a real-world task in corporate bond rating prediction and compare them to other approaches.

• Wow, thanks, this is perfect! This is very similar to my application as well! Can't believe I never found this paper. Jul 11 '18 at 17:01

You may want to have a look at "Unconstrained Monotonic Neural Networks".

The basic idea is to construct a neural network that forces the output to be positive. The integral of the output from that neural network is the final output that forms a monotonic function.

The paper describes how to train such a neural network. That is to say how to get the derivatives for the parameters with the added integral.

Here is the link to the paper: https://arxiv.org/pdf/1908.05164.pdf

• Welcome to the site. At present this is more of a comment than an answer. You could expand it, perhaps by giving a summary of the information at the link, or we can convert it into a comment for you.
– Sycorax
Apr 7 at 20:51