Deep NN validation. Possible underfitting?

Question

The objective of the NN is to classify instances (tuples of $50$ real values) as either 'signal' or 'background' (labeled as '0' or '1'). After scaling the tuples to contain values from $0$ to $1$, I build the model as:

model = Sequential([
    Dense(units=16, input_shape=(50,), activation='relu'),
    Dense(units=32, activation='relu'),
    Dense(units=2, activation='softmax')
])

I chose Adam with $0.0001$ learning rate and sparse categorical crossentropy as loss.

Then I fit the model with batch size $100$ and $30$ epochs (training and test set contain $36000$ instances each).

Then, the graphs of accuracy and loss are respectively:

Is my model underfitting and if so, what causes it? Should I use a different optimizer, activation function, loss function or number of epochs? Any suggestions for improving it are welcome.

Answer 1

It doesn't look like the model is underfitting from these graphs. The model is learning something, which can be seen from the decreasing validation loss. It is also not overfitting too badly, because the validation loss isn't deviating from the training loss too much, it's still decreasing.

Underfitting would be the model not picking up on a pattern that is in the dataset. You wouldn't be able to tell it from these graphs, except if, for some reason, you expect the model accuracy to get much higher, for example, because some more complicated model is getting better accuracy.

From general principles, I would say a model with 2 hidden layers is pretty complex, so underfitting wouldn't be a major concern. That said, a CNN with many layers is effective for MNIST, which has only 28x28=784 input values, so not that far off, but with many more parameters.