I'm trying to implement a simple neural network to fit a XOR function as shown in the book 'Deep Learning' by Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016). Here is my python code using Keras :
import keras
import numpy as np
# creating dataset
x = np.zeros((20000, 2), dtype=int)
x[5000:10000,1] += 1
x[10000:15000,0] += 1
x[15000:,:] += 1
np.random.shuffle(x)
y = [bit1 ^ bit2 for bit1, bit2 in x ]
x_train = x[:15000]
y_train = np.transpose(y[:15000])
x_test = x[15000:]
y_test = np.transpose(y[15000:])
# generating keras model
model = keras.models.Sequential([
keras.layers.Dense(2, input_shape=(2,)),
keras.layers.Dense(2, activation='relu'),
keras.layers.Dense(1, activation='relu')
])
# compiling using stochastic gradient descent and MSE
model.compile(optimizer='SGD',
loss='mean_squared_error')
# fit and evaluate
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)
The issue is that I'm stuck with a loss equal to 0.5, does anyone know what I'm missing here ?