2
$\begingroup$

I understand that the data type of the inputs would have to be the same, but say that I have all of the numbers as floats with two decimals. Would a neural network be able to learn the classification:

X      y
100.00 1
100.09 0
100.70 0
78.00  1
78.17  0
...

Intuitively I think that it wouldn't be an easy task. Perhaps it is trivial but then I'm missing how it would work. Suprisingly some googling hasn't given any results (may be I'm not using the right terms...)

Edit: as per the comments below, in order to narrow down the question, let's suppose the Xs are stored as float variables defined in Python.

Thanks!

$\endgroup$
10
  • 3
    $\begingroup$ It's unclear what a "float with two decimals" actually is: would you be storing these as IEEE double-precision values? As text strings? As integers equal to 100 times their values? With some of these possibilities (especially the strings) the task ought to be much easier. $\endgroup$
    – whuber
    Commented Mar 3, 2021 at 15:11
  • $\begingroup$ mm say that I store them as double-precision values. Would it make it difficult for the NN to differentiate? $\endgroup$
    – RR_28023
    Commented Mar 3, 2021 at 15:13
  • $\begingroup$ Do you mean something like x=float(78.17) and y=float(100.00) using Python code? $\endgroup$
    – Dave
    Commented Mar 3, 2021 at 15:14
  • 3
    $\begingroup$ I believe that storing them as double-precision floats might make the problem as difficult as possible ;-), unless you are passing the 64-bit sequences to the classifier (as 64 binary features). It looks like "feature engineering" may be a key part of the problem and its solution. $\endgroup$
    – whuber
    Commented Mar 3, 2021 at 15:18
  • 2
    $\begingroup$ Just spitballing, but I think this problem reduces to a whether an NN that can determine if $x \mod 1 > 0$ is true or false, so the answer turns on whether an NN can approximate the modulus function. $\endgroup$
    – Sycorax
    Commented Mar 3, 2021 at 18:55

1 Answer 1

4
$\begingroup$

No, I don't think this is possible in a vanilla (no feature engineering) approach.

Any neural network with continuous activations will produce a continuous function at the end. Given a training set as you describe, it is reasonably likely that a network with enough neurons and, say, two layers will perfectly classify the training set: learn triples of first-layer neurons like $\operatorname{relu}(x-1.99)$, $\operatorname{relu}(x-2)$, and $\operatorname{relu}(x-2.01)$, then combine them (the first minus twice the second plus the third?) in a second-layer neuron to get a single spike around $x=2$. This can happen around every integer in your training set, and nearby non-integers will regulate the width of the spikes. (Although, as usual, in practice I can't currently get the neural net to learn its way into this structure. I'll keep playing a little bit and update with a notebook if I'm successful.)

But, for any integer not in your training set, the network has no reason to deviate from this structure and give (significantly) positive values anywhere else. And outside the training range, all bets are off for even the non-integers. You would need some mechanism to encourage periodicity, but if you do that then the problem becomes (almost) trivial.


I got a bit of improvement in my attempts to train into the explicit network described above. It's still rather dependent on the initial weights, and there are a couple of points it doesn't quite learn, but I lost interest in cleaning it up, so here it is:

notebook link

trained NN predictions

The training data has each digit except 6 and then random non-integers in $(0, 10)$. So there's nothing to enforce anything about negative numbers, and the integer 6 is unnoticed by the network. (But the model has failed to understand 1, 2, 3, and seems to have superfluous kinks in the negative range.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.