Training/validation error remains the same across epoch

Question

I'm trying to train a neural network on the following dataset, and I see that the training and the validation error do not change across the epoch. I have tried changing the number of input/hidden layer units, dropout rate and other hyper parameters, but to no avail.I'm hoping someone helps me figure out why this is the case. Here is the entire code :

library(keras)
library(caret)
data = read.csv("/home/draxler/Desktop/Churn_Modelling.csv")
data = data[,-c(1,2,3)]
levels(data$Gender) = c("0","1")
levels(data$Geography) = c("0","1","2")

temp = preProcess(data[,c("Balance","EstimatedSalary")], method = c("center","scale"))
data[,c("Balance","EstimatedSalary")] = predict(temp,data[,c("Balance","EstimatedSalary")])

ind = sample(1:nrow(data), .7*nrow(data))

train = data[ind,]
test = data[-ind,]

train_x = as.matrix(train[,-11])
train_y = to_categorical(train[,11])

test_x = as.matrix(test[,-11])
test_y = to_categorical(test[,11])


model = keras_model_sequential()

model %>% layer_dense(512, input_shape = 10,activation = "relu") %>%
  layer_dropout(.3) %>% 
  layer_dense(unit = 64,activation = "relu") %>%
  layer_dropout(.3) %>% 
  layer_dense(2, activation = "softmax")

model %>% compile(
  loss = 'categorical_crossentropy',
  optimizer = 'adam',
  metrics = c('accuracy')
)


model %>% fit(train_x, train_y, epochs = 100, batch_size = 128, validation_split = 0.3,verbose = 2)

x = model %>% evaluate(train_x,train_y,batch_size = 128)

Here I've plotted the training/validation loss and training/validation accuracy.

Answer 1

It seems that you neural network has already converged to a solution. You should see what is the best performance other people have achieved on the same dataset (since it's on Kaggle).

For a problem like this, there is probably an upper bound for the accuracy that is below 1.0, since medical diagnosis has inherent uncertainties that can not be fully explained by the data. Looking at the dataset I believe this to be the case and think our performance is already pretty good.