I am trying to resolve the xor operator using neural networks, and to accomplish that this is my code:

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([[0], [1], [1], [0]])

model = Sequential([
    Dense(2, activation="sigmoid", input_dim=2),
    Dense(1, activation="sigmoid")

model_1.compile(loss="binary_crossentropy", optimizer="adamax")
model_1.fit(X, y, batch_size=4, epochs=16000)

model_2.compile(loss="binary_crossentropy", optimizer="sgd") # Never converge independently of how many epochs
model_2.fit(X, y, batch_size=4, epochs=16000)

So, can someone explain why using SGD the model does not converge? Why in this case using adamax is better than SGD? And when to use sgd or adamax.

  • 1
    $\begingroup$ it converges, just much slower, try larger learning rate optimizer=SGD(lr=0.5) $\endgroup$ – itdxer May 29 '19 at 15:37
  • $\begingroup$ Thank you, @itdxer! $\endgroup$ – Caaarlos May 29 '19 at 16:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.