New answers tagged keras
1
I would say that your learning rate is too high. The Adam optimizer can become unstable in areas where the gradient is very low. In this case, you have a very small network and you may be close to an optimum in a very flat region of the gradient.
Try lowering the learning rate and see if the oscillation goes away, or try switching to a different optimizer.
0
Note that there can be the effect called double decent, where additional training still helps your performance on the test set, although your training error is already well below the test error:
https://openai.com/blog/deep-double-descent/
0
Your example is just broadcast semantics. This is easy to do using PyTorch, and probably Keras or Tensorflow (but I don't use them).
x = torch.FloatTensor([[[0.0, 1.0], [0.0, 0.0], [1.0, 0.0], [0.0, 1.0]]])
y = torch.FloatTensor([[[0.43913543], [0.0], [-1.3466451], [0.43913543]]])
z = x * y
print(z)
Yields
tensor([[[ 0.0000, 0.4391],
[ 0.0000, 0.0000]...
Top 50 recent answers are included