# Feedforward Neural networks for Regression confusion

I’m a bit confused about the concept of using feedforwrd neural networks via backpropagation to model a nonlinear relationship between the input and output variable in a regression setting. Can this be done? I.e., could such an ANN deduce a relationship like $y = \sin(x^2) + x^3$?

If it’s not easy to do this, does it have to do with the fact that in regression problems the output layer activation function is linear?

Thanks.

• If there are non linearities in earlier layers then it doesn’t really matter if the last layer is linear AFAIK – kbrose Aug 29 '18 at 23:21

For example, suppose we want to fit your example function. There's a single input $x \in \mathbb{R}$. We feed it through one or more nonlinear hidden layers, each containing multiple units. The output of the last hidden layer (with $p$ units) is $h(x) = [h_1(x), \dots, h_p(x)]$. There's a single, linear output unit with weights $w = [w_1, \dots, w_p]$ and bias $b$. So, the output of the entire network is:
$$f(x) = b + \sum_{i=1}^p w_i h_i(x)$$