# Activation function for output layer for unbounded values

Are there activation functions for the output layer of a neural network whose range is $\mathbb{R}$, so that I can perform true function approximation (and not be limited to a arbitrary bounds on the output)?

I've considered using $\tanh^{-1}(2x -1)$ on the output of a sigmoid neuron since $\tanh^{-1}(x)$ has a domain of (-1, 1) and a range of $\mathbb{R}$. The problem is that $\tanh^{-1}(2\sigma(x) -1)$ is equivalent to $x/2$ (see this, so I'm not actually getting a truly unbounded function- the output of this kind of neuron is limited to [half the sum of all negative weights, half the sum of all positive weights].

Are there different combinations of output functions that can give me a truly unbounded output?

• How do propose to train such a model, given that your training data is finite? I mean, are you doing this as a theoretical exercise on a known function? – P.Windridge Oct 29 '17 at 14:20

There's identity activation function. It'll simply output your $a^{[l]}=z^{[l]}$, where $z^{[l]}=\beta+w \cdot a^{[l-1]}$
• I'm not looking for a way to have an arbitrarily large bound on the output, I'm looking for the output to have a range of $\mathbb{R}$. It sounds like with a linear activation function, the range can only be as large as the input weights. That doesn't satisfy what I'm trying to do. – PixelArtDragon Mar 9 '17 at 23:21