Estimate monotone function from noisy obsersations

Question

Assume that $y=f(x)$ is an unknown monotonically increasing function of variable $x$. We have access to $N$ observations of this function given as tuples $(x_i,y_i)$, such that $y_i=f(x_i)$. The problem is to estimate $f(.)$ from this observations. In the noise-less case, they obey the monotone condition that $f(x_i) < f(x_j)~,~\forall x_i < x_j$. In the noisy case, some of the observed samples may no longer obey the monotone condition though the majority might be. I am unfamiliar with this kind of problems. Is this specific problem studied in literature? If not, can you point me to the techniques which are used in similar kind of problems?

I did a bit of my own research and Isotonic regression seems like a candidate to me. Is that right?

Answer 1

Yes Isotonic regression is a classical solution when the output is continuous. This however creates a rather non smooth function: https://en.wikipedia.org/wiki/Isotonic_regression.

When the output is discrete, then you can use ordinal regression. If you discretize $Y$ first (define interval slices), you can still use it. And then interpolate.

Finally you can use a parametric method. Define a set of functions with parameters $a,b,c...$ that are increasing for any parameter. Then fit with least squares (or maybe better: with canonical GLM to have no bias).

This is an example of fitting with such functions:

• red: raw noisy data
• yellow: $f(x)=a^2x^{b^2}$
• green : $f(x)=(a^2x+b^2)\left(1-\frac{1}{1+c^2x}\right)$.

(using squares everywhere is a trick to force positive values)

I would naturally use the parametric method is most situations I've known because this creates a very smooth interpretable curve and it is very easy to implement.