I'm trying to train a neural network on the following dataset, and I see that the training and the validation error do not change across the epoch. I have tried changing the number of input/hidden layer units, dropout rate and other hyper parameters, but to no avail.I'm hoping someone helps me figure out why this is the case. Here is the entire code :
library(keras)
library(caret)
data = read.csv("/home/draxler/Desktop/Churn_Modelling.csv")
data = data[,-c(1,2,3)]
levels(data$Gender) = c("0","1")
levels(data$Geography) = c("0","1","2")
temp = preProcess(data[,c("Balance","EstimatedSalary")], method = c("center","scale"))
data[,c("Balance","EstimatedSalary")] = predict(temp,data[,c("Balance","EstimatedSalary")])
ind = sample(1:nrow(data), .7*nrow(data))
train = data[ind,]
test = data[-ind,]
train_x = as.matrix(train[,-11])
train_y = to_categorical(train[,11])
test_x = as.matrix(test[,-11])
test_y = to_categorical(test[,11])
model = keras_model_sequential()
model %>% layer_dense(512, input_shape = 10,activation = "relu") %>%
layer_dropout(.3) %>%
layer_dense(unit = 64,activation = "relu") %>%
layer_dropout(.3) %>%
layer_dense(2, activation = "softmax")
model %>% compile(
loss = 'categorical_crossentropy',
optimizer = 'adam',
metrics = c('accuracy')
)
model %>% fit(train_x, train_y, epochs = 100, batch_size = 128, validation_split = 0.3,verbose = 2)
x = model %>% evaluate(train_x,train_y,batch_size = 128)
Here I've plotted the training/validation loss and training/validation accuracy.