In this introduction to neural networks (I enjoy it because it builds a digit-recognition neural network from scratch with just numpy, without any high-level NN library like pytorch or tensorflow; thus it helps to understand the internals), we import images from the well-known MNIST digits dataset, but instead of mapping the pixels grayscale to [0.00, 1.00], it does it to [0.01, 0.99]:
We map the values of the image data into the interval [0.01, 0.99] by dividing the train_data and test_data arrays by (255 * 0.99 + 0.01)
Then we create the one hot representation of digits, but again, avoiding 0.00 and 1.00:
We are ready now to turn our labelled images into one-hot representations. Instead of zeroes and one, we create 0.01 and 0.99, which will be better for our calculations:
lr = np.arange(no_of_different_labels) # transform labels into one hot representation train_labels_one_hot = (lr==train_labels).astype(np.float) test_labels_one_hot = (lr==test_labels).astype(np.float) # we don't want zeroes and ones in the labels neither: train_labels_one_hot[train_labels_one_hot==0] = 0.01 train_labels_one_hot[train_labels_one_hot==1] = 0.99 test_labels_one_hot[test_labels_one_hot==0] = 0.01 test_labels_one_hot[test_labels_one_hot==1] = 0.99
I tried both:
- with [0.00, 1.00] mapping => 94.0% accuracy
- with [0.01, 0.99] mapping => 94.5% accuracy
so it seems to confirm that this little trick improves a little bit the accuracy.
Why and how does this work?