What is the difference between a loss function and an error function? Is the term "loss" synonymous with "error"? Is there a difference in definition?
Also, what is the origin of the term "loss"?
NB: The error function mentioned here is not to be confused with normal error.
 A: In the context of a predictive or inferential model, the term "error" generally refers to the deviation from an actual value by a prediction or expectation of that value.  It is determined entirely by the prediction mechanism and the actual behaviour of the quantities under observation.  The "loss" is a quantified measure of how bad it is to get an error of a particular size/direction, which is affected by the negative consequences that accrue for inaccurate prediction.
An error function measures the deviation of an observable value from a prediction, whereas a loss function operates on the error to quantify the negative consequence of an error.  For example, in some contexts it might be reasonable to posit that there is squared error loss, where the negative consequence of an error is quantified as being proportional to the square of the error.  In other contexts we might be more negatively affected by an error in a particular direction (e.g., false positive vs. false negative) and therefore we might adopt a non-symmetric loss function.
The error function is a purely statistical object, whereas the loss function is a decision-theoretic object that we are bringing in to quantify the negative consequences of error.  The latter is used in decision theory and economics (usually through its opposite - a cardinal utility function).

An example: You are a criminal racketeer running an illegal betting parlour for the Mob.  Each week you have to pay 50% of the profits to the Mob boss, but since you run the place, the boss relies on you to give a true accounting of the profits.  If you have a good week you might be able to stiff him out of some dough by underrepresenting your profit, but if you underpay the boss, relative to what he suspects is the real profit, you’re a dead man.  So you want to predict how much he expects to get, and pay accordingly.  Ideally you will give him exactly what he is expecting, and keep the rest, but you could potentially make a prediction error, and pay him too much, or (yikes!) too little.
You have a good week and earn $\pi = \$40,000$ in profit, so the boss is owed $\tfrac{1}{2} \pi = \$20,000$.  He doesn’t know what a good week you had, so his true expectation of his share is only $\theta = \$15,000$ (unknown to you).  You decide to pay him $\hat{\theta}$.  Then your error function is:
$$\text{Error}(\hat{\theta}, \theta) = \hat{\theta} - \theta ,$$
and (if we assume that loss is linear in money) your loss function is:
$$\text{Loss}(\hat{\theta}, \theta) = \begin{cases}
\infty & & \text{if } \hat{\theta} < \theta \quad \text{(sleep wit' da fishes)} \\[6pt]
\hat{\theta} - \pi & & \text{if } \hat{\theta} \geqslant \theta \quad \text{(live to spend another week)} \\
\end{cases} $$
This is an example of an asymmetric loss function (solution discussed in the comments below) which differs substantially from the error function.  The asymmetric nature of the loss function in this case stresses the catastrophic outcome in the case where there is underestimation of the unknown parameter.
