# How many hidden neurons are necessary to approximate a linear function?

Let's assume we have a multilayer feedforward network with one hidden layer. The hidden layer must consist of at least two neurons with logistic activation function. How many hidden neurons are necessary in order to approximate a linear function $$f(x) = \beta_0 + \beta_1 x + \epsilon$$ with an accuracy similar to a linear regression model?

On the other hand, linear functions are linear everywhere, so the OLS model could do well on any interval if the true function is linear, or poorly if it is not. For example, you can approximate the linear part of $$\tanh$$ well with a line, or any of its asymptotes, but the linear approximation is poor if you consider the entire function.
It's not possible to make a general statement relating this to OLS, because different OLS models will achieve different levels of precision, depending on the problem. If the true model is $$y = \beta_0 + \beta_1 x + \epsilon$$, then the expected MSE depends on the distribution of $$\epsilon$$. Are the $$\epsilon$$s independent, identical realizations from some distribution? Which distribution? Or something else?