Are there activation functions for the output layer of a neural network whose range is $\mathbb{R}$, so that I can perform true function approximation (and not be limited to a arbitrary bounds on the output)?
I've considered using $\tanh^{-1}(2x -1)$ on the output of a sigmoid neuron since $\tanh^{-1}(x)$ has a domain of (-1, 1) and a range of $\mathbb{R}$. The problem is that $\tanh^{-1}(2\sigma(x) -1)$ is equivalent to $x/2$ (see this, so I'm not actually getting a truly unbounded function- the output of this kind of neuron is limited to [half the sum of all negative weights, half the sum of all positive weights].
Are there different combinations of output functions that can give me a truly unbounded output?