The universal approximation theorem states that a feedforward neural network (NN) with a single hidden layer can approximate any function over some compact set, provided that it has enough neurons on that layer.
This suggests that the number of neurons is more important than the number of layers.
But in practice deep learning is obviously very successful at various prediction tasks. Why is that? Shouldn't all deep NNs be equivalent to single layered NNs with enough neurons? Why do we need depth?