I'm encountering an issue where a classifier I'm developing reports validation errors during training that span a wide range of values without consistently decreasing over time. Unfortunately, I'm new to ML and related topics and can't seem to diagnose the problem as a result.
The base code of the classifier comes from the theano deep_learning tutorials, specifically from the multilayer perceptron tutorial: link
I have changed the code from the tutorial in two ways: (1) Altered the data being used (2) Altered the topography of the MLP to account for a different number of input and output nodes
Here is an image of typical results:
The data I'm using comes from a wine quality experiment: link
I would appreciate help understanding:
(1) Why this is happening (2) How I should go about fixing it