Why isn't (symmetric) log(1+x) used as neural network activation function? Specifically, I mean
$$
f(x)=
\begin{cases}
-\log(1-x) & x \le 0 \\
\space \space \space \log(1+x) & x \gt 0 \\
\end{cases}
$$
Which is red in the plot:

It behaves similarly to widely used $\tanh(x)$ (blue) except it avoids saturation/vanishing gradients since it has no horizontal asymptotes. It's also less computationally expensive.
Is there some issue with it I'm missing?
 A: For a long time, neural network researchers believed that sigmoid activations like the inverse logit and $\tanh$ were the only activations that were necessary. This is because the Cybenko (1989) Universal Approximation Theorem (loosely) states that, under certain conditions, a neural network can approximate certain functions to a desired level of precision with 1 hidden layer & a finite number of units. One of the conditions is that the activation function is bounded. (For full details, consult the paper.)
The function $f(x)=\log(1+x)$ is not bounded, so it does not satisfy the boundedness condition.
However, in the time since Cybenko published his UAT, many other UAT variations have been proven in different settings & allowing more flexibility in the choice of activation functions, number of layers, and so on.
From the perspective of modern neural network theory, you would need to show that the proposed activation has some desirable property that is not found in alternative choices. One problem that I anticipate with this activation is that its derivative is $f^\prime(x)=\frac{1}{1+|x|}$, which is strictly between 0 and 1 almost everywhere. This is undesirable because of the vanishing gradient phenomenon. An activation function with derivative exactly 1 for a "large" portion of inputs is preferable because it ameliorates the vanishing gradient.

Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control Signal Systems 2, 303–314 (1989). https://doi.org/10.1007/BF02551274
