Can a neural network learn "a == b" and "a != b" relationship, with limited data? For example, I have the following feature set:
{a:1, b:1} -> 1
{a:2, b:2} -> 1
{a:3, b:3} -> 1

{a:1, b:2} -> 0
{a:4, b:6} -> 0
{a:3, b:5} -> 0

And some more data points.
I don't give features like a == b or a != b.
Can a neutral network get the following logic?
(a == b)  ->  1
(a != b)  ->  0

Assume I give inputs like
{a:123, b:123}
{a:666, b:668}

The model never sees inputs such as 123 or 666 before.
 A: You function can be represented as:
$$f(x,y) = \lim_{n\to+\infty}(\sigma(1/|x-y|))^n$$
A good approximation can be obtained with a large $n$.
A neural network can approximate further that function. In fact, depending on what you consider a neural network to be, the function itself is already a neural network with special activation functions.
A: The absolute value function can be written as
$$|a-b|=\text{ReLU}(a-b) + \text{ReLU}(b-a),$$ and has a minimum at 0 for $a = b$.  We can compose this with a sigmoid layer
$$\sigma\left(\delta\left(\text{ReLU}(a-b) + \text{ReLU}(b-a)+\epsilon  \right) \right),$$ and this is very close to what is desired for $\epsilon < 0$ because this shifts the minimum below 0 and choosing $\delta < 0$ means that a negative value maps to a number greater than 0.5, and a positive value to a number less than 0.5.
Naturally, there will be "wrong answers" for $|a-b|$ that are close to $\epsilon$. This is unavoidable with continuous functions (such as those used in neural networks). Changing the magnitude of $\epsilon$ controls this.
A deficiency with this is that its outputs are not exactly 0 and exactly 1. You can't obtain these values exactly because the sigmoid function only obtains 0 and 1 in a limit of infinitely large or small values. It's probably also hard to train a neural network to find weights that work well, especially for representing $|a-b|$.
