I am learning about NN and gave myself a problem to solve, which is basically the typical "ceiling" function:

a.b = a+1 when b != 0.
a.b = a when b = 0.
where a.b is a floating point number, with a and b being integers between 0 and 9.

I have made a few attempts, but my NN doesn't solve the problem. It gets semi-close and then just swirls.

My inputs are the two integers a and b. My output layer is a single node n. I have tried a few variations of hidden hiddens including 1 layer of 2, 4, 5, and 10 neurons, and two layers of 2, 5, and 10 neurons. None seem to matter.

My learning approach is pure Fisher-Price genetic.

EDIT - getting close! Input on the top, bottom is ceil(x).

0.00 0.30 0.60 0.90 1.20 1.50 1.80 2.10 2.40 2.70 3.00 3.30 3.60 3.90 4.20 4.50 4.80 5.10 5.40 5.70 6.00 6.30 6.60 6.90 7.20 7.50 7.80 8.10 8.40 8.70 9.00 9.30 9.60 9.90 
0.00 0.99 0.99 0.99 2.00 2.00 2.00 2.99 2.99 2.99 2.99 4.00 4.00 4.00 5.00 5.00 5.00 6.00 6.00 6.00 6.00 7.01 7.01 7.01 8.01 8.01 8.01 9.02 9.02 9.02 8.99 10.02 10.02 10.02 

Can a neural network solve this problem?

Yes. A simple neural network with three units and a ReLU activation function can represent this function exactly:

$$ y=[1;1;-1] \cdot \sigma\left( \begin{bmatrix}1&0&0\\0&1&0\\0&1&-1\end{bmatrix}\begin{bmatrix}a\\b\\1\end{bmatrix}\right) $$

This network represents a function $y=a+\sigma(b) - \sigma(b-1)$, which for your input range performs rounding.

Is a neural network an appropriate tool to use for rounding?

Absolutely not. If there exists a simple, known math relationship that can be evaluated efficiently, there is no need to use neural networks.

Is this problem an interesting case for studying behavior of neural networks?

Sort of. It is small enough to evaluate the network on all cases at each timestep, which might give certain insights into the learning behavior. As you can see on your own experiments, even though there exists a trivial solution to the problem (above), your learning algorithm fails to discover it. That's certainly one of interesting aspects of neural network learning.

  • $\begingroup$ I know there are better tools than a NN for this :-) I was wondering if there was something about the problem itself that made NNs unable to do it. $\endgroup$ – Tony Ennis Jun 17 '19 at 15:42
  • $\begingroup$ What is a "unit", please? An integer input? $\endgroup$ – Tony Ennis Jun 17 '19 at 15:44
  • 1
    $\begingroup$ Network has layers (in this case one hidden and one output), and each layer has units (hidden layer has three, output layer has one). A "unit" performs a dot product of two vectors (input*weights) and applies an activation function. Weights of one unit form rows of the weight matrix above. The extra "1" input is the bias term, effectively the last column of the weight matrix can be considered the bias vector. $\endgroup$ – Jan Kukacka Jun 17 '19 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.