Assume we have a neural network with 2 input neurons, 1 hidden neuron and 1 output neuron. The output of such a network is calculated as follows:
$$ f(w_1x_1 + w_2x_2)w_3$$ , where $f$ is the activation function of the hidden neuron, $x_1$ and $x_2$ are the network inputs, $w_1$ and $w_2$ the connection weights from the input neurons to the hidden neuron and $w_3$ the connection weight from the hidden neuron to the output neuron.
If $f$ is a linear activation function, the above neural network is identical to a linear model with coefficients $w_1w_3$ and $w_2w_3$.
Is it possible to derive the coefficients of a linear model if the activation function $f$ is a sigmoid or tanh function? If yes, how?