I am a beginner in machine learning. I have built a logistic classifier in Python using TensorFlow to train on notMNIST dataset. My code is as such:
weights = tf.Variable(tf.truncated_normal(shape = [784, 10]))
bias = tf.Variable(tf.zeros(shape = [10]))
logits = tf.matmul(features, weights) + bias
prediction = tf.nn.softmax(logits)
cross_entropy = -tf.reduce_sum(labels * tf.log(prediction), reduction_indices=1)
loss = tf.reduce_mean(cross_entropy)
train_feed_dict = {features: train_features, labels: train_labels}
valid_feed_dict = {features: valid_features, labels: valid_labels}
test_feed_dict = {features: test_features, labels: test_labels}
is_correct_prediction = tf.equal(tf.argmax(prediction, 1), tf.argmax(labels, 1))
accuracy = tf.reduce_mean(tf.cast(is_correct_prediction, tf.float32))
epochs = 5
batch_size = 50
learning_rate = 0.1
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)
validation_accuracy = 0.0
with tf.Session() as session:
session.run(tf.global_variables_initializer())
batch_count = int(math.ceil(len(train_features)/batch_size))
for epoch_i in range(epochs):
for i in range(batch_count):
session.run(optimizer, feed_dict = train_feed_dict)
print(session.run(accuracy, feed_dict = train_feed_dict))
However, the problem is that while the training loss is decreasing continuously, the accuracy wavers initially, and then finally stagnates (at around 0.062). I am not able to understand what's wrong with the code. Any help would be appreciated. Thanks.
