An autoencoder attempts to reconstruct the input. During the process it could learn the identity function if number of hidden layers are equal or more than number of inputs. However that is not desirable. During learning, the autoencoder discovers the most common features in the input. For example if the input is a natural image, it discovers an edge because it is the most common feature in all natural images. In the simplest case, the autoencoder is constructed with fewer hidden units than its input layer. As hidden units are added, it can enlist more features to represent the input. However, as number of hidden units exceeds the number of input units, the features becomes more and more dependent. The autoencoder can discover those features when the hidden layers are densely activated. If sparsity is introduced to network, it restricts activation of active hidden units and reduces the dependency between features, while increasing the number of features, which is desirable.