I have a NN that I would like to square a number. This is a learning exercise for me.

My input is the number to be squared, the output is the square.

Two questions: 1) How can this possibly work? The weights and nodes of the NN need to square to a number that isn't fixed.

2) Assuming I am wrong, what is a strategy for choosing the numbers of nodes and layers for a NN?

  • 1
    $\begingroup$ As an example: stats.stackexchange.com/questions/299915/… but a necessary unstated component t to your question is what amount of precision you want in the result; the universal approximation theorem lays out technical criteria for NNs to approximate specific functions. $\endgroup$ – Reinstate Monica Jun 24 at 18:29

The ReLU activation function should take care of this.

ReLU works by fitting short, straight lines to approximate curves. That should be able to create a parabola. You will have performance suffer for inputs with very large absolute values, but we know that models won't be perfect.

I was thinking that one hidden layer could take care of this, but reading about the universal approximation theorem (which I suggest doing), we can be more efficient by having fewer nodes in multiple hidden layers than tons of nodes in one hidden layer.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.