Is there a universal approximation theorem for monotone functions?

The universal approximation theorem basically states that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of $$\mathbb{R}^n$$.

Let's define a monotone function of vectors to mean any real-valued $$f$$ such that for any two vectors $$\mathbf{x}$$ and $$\mathbf y$$, $$x_i \le y_i \forall i \implies f(\mathbf{x}) \le f(\mathbf{y}).$$ The vector-valued function case is analogous.

Is it true that a feed-forward network with a single hidden layer containing a finite number of neurons, and whose link weights are nonnegative (but whose biases remain unconstrained) can approximate continuous monotonic functions on compact subsets of $$\mathbb{R}^n$$?

• I don't know the answer but Kurt Hornik ( part of R-core ) is a generous, knowledgable person who will either answer you directly or atleast point you in the right direction. He wrote the famous paper with Hal White whose title escapes me. – mlofton Nov 10 '18 at 6:30

I am not sure if a single hidden layer is sufficient, but it can be shown that if your input is in $$\mathbb{R}^k$$, you will need at most $$k$$ hidden layers. See Theorem 3.1 in https://ieeexplore.ieee.org/document/5443743
Theorem 3.1: For any continuous monotone nondecreasing function $$f : K \rightarrow \mathbb{R}$$, where $$K$$ is a compact subset of $$\mathbb{R}^k$$, there exists a feedforward neural network with at most $$k$$ hidden layers, positive weights, and output $$O$$ such that $$|f(\mathbf{x}) - O_{\mathbf{x}}| < \varepsilon$$, for any $$\mathbf{x} \in K$$ and $$\varepsilon > 0$$.