What do these weights learn?
At the most literal level, they learn to predict the tokens of a sequence of text, given a corrupted form of that same sequence. In other words, they learn common patterns in language. BERT is a gigantic pseudo–language model.
Are weights learned for each token?
It’s easy for a misunderstanding to arise here. For the same word type occurring in different sentences (or in different places within the same sentence), the final representation will be different. Still, there are not parameters for each word token.
There are parameters for each word type, just like you’d find in a non-contextualized embedding (e.g. word2vec). The type embeddings of each token are passed through the BERT encoder network to produce vector representations of each token in context.
how [are] the pretrained weights helping?
TL;DR better initialization.
There are many local optima in the loss surface for large neural networks. Gradient-based optimization can converge to any of these, and which one you find depends on the initialization of the network. Some local optima are better than others. So how do we pick the right initialization, to give us a good local optimum? Especially with the vast number of parameters (and therefore, choices) in these models, we’ll take all the help we can get!
The intent of pretraining (and sequential transfer learning) is better initialization. If a parameter setting is better for task A (e.g. the BERT training objective), we suspect that it is close to a good parameter setting for task B (e.g. sentence classification). Start training on task B at the parameters you learned for task A.
Empirically, this approach seems to work well. The vast amount of training data for task A helps to get good performance on the related task B, which usually doesn’t have nearly as much data. If we just used the data for task B, we wouldn’t find as suitable of an optimum.