Imagine we are working with the MNIST dataset and creating a neural network with 1 hidden layer. So we have a vector of 784 inputs, 100 hidden nodes, and 10 outputs.
If we were to visualize each node in the hidden layer (the wTx of inputs for each node) would we see features of the data? Such as circles, semi-circles, lines, etc?
And this is dimensionality reduction? Going from 784 pixels to 100 features