What does "learn the linear part of a mapping" mean? In the paper "Efficient BackProp" , the authors talk about initializing the weights  not too small and not too large:

Intermediate weights that range over the sigmoid's linear region have the advantage that (1) the gradients are large enough that learning can proceed and (2) the network will learn the linear part of the mapping before the more difficult nonlinear part.

What do they mean by "learn the linear part of the mapping first"? And why is this a good thing?
 A: What does it even mean?
As whuber already mentioned, look at the function graph of a sigmoid :

What stands out are the parts between [-1,+1], which are approximately linear, and the parts before/beyond that.
If you look at this kind of mapping, then the linear region is the steepest; mathematically it has the largest derivative. This also means a change in the input values/argument values creates the largest change in output values. 
Which means, you want to use mostly this part. In a neural network, the output of this kind of function is input to another node, usually summing several input nodes. Since one knows that the function values is limited to [0, +1], it is straightforward to set the initial weight so that "on average" the linear part is used.
In short: since all that is changed in "standard backpropagation" are the weights, one aims at "hitting" the linear spot of the sigmoids in the beginning/at the initialization stage.
Edit; add second part of the question:
What does "learn the linear part of the mapping" mean?
For this question you have to know a bit about what a neural network is: in principal it is just a function (or mapping) from the input space to the output space.
If you are familiar with functions, then you have several possiblities to interpret them.
On of the key parts of ANNs is the fact they represent a class of functions that is capable of approximating any function.
And the process of approximating is called "learning" or "training".
I don't really like analogies, but here it might be helpful. So the following statement is not exactly true:
Think that ANNs behave like polynomials (under certain circumstances).
If you want to approximate an unknown function with a polynomial, then there are lots of ways to do it.
Most approximations in application end at the linear term, where there are even more ways to find clever solutions to it.
The main reason is that linear approximations have very nice properties and are relatively easy to calculate.
If you then also think about the Taylor series expansion, then you'll see that the elements are also polynomials.
Constant + linear + quadratic + ... and so on.
The problem, and more specifically its numerical solution becomes increasingly more difficult, or expensive to compute.
To summarize:
An ANN is a mapping of the input to the output space.
It can be decomposed (represented) by a polynomial expansion series.
The lower the order of the exponential, the less expensive/complex the elements are to compute.
