I am new to deep learning, so this might be a trivial question. But I am wondering why deep learning (or neural network) does not work very well on small labeled data. Whatever research papers I have read, their datasets are huge. Intuitively that's not surprising because our brain takes a lot of time to train itself. But is there a mathematical proof or reason why neural network does not work well in such cases ?
The neural networks used in typical deep learning models have a very large number of nodes with many layers, and therefore many parameters that must be estimated. This requires a lot of data. A small neural network (with fewer layers and fewer free parameters) can be successfully trained with a small data set - but this would not usually be described as "deep learning".