Is there a way or theorem that allows me to constrain a function $N(x)$, where $N(x)$ is a feedforward mlp such that it will always be a monotonic function. The simplest way is of course to use a penalty method, but this does not 100% guarantee, I am looking to see if there is a function or transformation that can be applied to $N(x)$ that constrains its form to being monotonic. The only way I can currently think of is by making all weights $w_{i,j,k} >0$ and that the activation function $f(a)$ are monotonic and non-negative (using $f(a) = \frac{tanh(a)+1}{2}$) and having a zero bias.


Here's an example of an early publication in this vein.

Joseph Sill. "Monotonic Networks". California Institute of Technology. 1998.

Monotonicity is a constraint which arises in many application domains. We present a machine learning model, the monotonic network, for which monotonicity can be enforced exactly, i.e., by virtue of functional form. A straightforward method for implementing and training a monotonic network is described. Monotonic networks are proven to be universal approximators of continuous, differentiable monotonic functions. We apply monotonic networks to a real-world task in corporate bond rating prediction and compare them to other approaches.

  • 1
    $\begingroup$ Wow, thanks, this is perfect! This is very similar to my application as well! Can't believe I never found this paper. $\endgroup$
    – Sam Palmer
    Jul 11 '18 at 17:01

You may want to have a look at "Unconstrained Monotonic Neural Networks".

The basic idea is to construct a neural network that forces the output to be positive. The integral of the output from that neural network is the final output that forms a monotonic function.

The paper describes how to train such a neural network. That is to say how to get the derivatives for the parameters with the added integral.

Here is the link to the paper: https://arxiv.org/pdf/1908.05164.pdf

  • $\begingroup$ Welcome to the site. At present this is more of a comment than an answer. You could expand it, perhaps by giving a summary of the information at the link, or we can convert it into a comment for you. $\endgroup$
    – Sycorax
    Apr 7 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.