# Is there a universal approximation theorem for monotone functions?

The universal approximation theorem basically states that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of $$\mathbb{R}^n$$.

Let's define a monotone function of vectors to mean any real-valued $$f$$ such that for any two vectors $$\mathbf{x}$$ and $$\mathbf y$$, $$x_i \le y_i \forall i \implies f(\mathbf{x}) \le f(\mathbf{y}).$$ The vector-valued function case is analogous.

Is it true that a feed-forward network with a single hidden layer containing a finite number of neurons, and whose link weights are nonnegative (but whose biases remain unconstrained) can approximate continuous monotonic functions on compact subsets of $$\mathbb{R}^n$$?

• I don't know the answer but Kurt Hornik ( part of R-core ) is a generous, knowledgable person who will either answer you directly or atleast point you in the right direction. He wrote the famous paper with Hal White whose title escapes me. Commented Nov 10, 2018 at 6:30

I am not sure if a single hidden layer is sufficient, but it can be shown that if your input is in $$\mathbb{R}^k$$, you will need at most $$k$$ hidden layers. See Theorem 3.1 in https://ieeexplore.ieee.org/document/5443743
Theorem 3.1: For any continuous monotone nondecreasing function $$f : K \rightarrow \mathbb{R}$$, where $$K$$ is a compact subset of $$\mathbb{R}^k$$, there exists a feedforward neural network with at most $$k$$ hidden layers, positive weights, and output $$O$$ such that $$|f(\mathbf{x}) - O_{\mathbf{x}}| < \varepsilon$$, for any $$\mathbf{x} \in K$$ and $$\varepsilon > 0$$.