My notes define a loss function as the 'cost' incurred when the true value of $\theta$ is estimated by $\hat\theta$. What kind of cost is it talking about? monetary cost? or is it something related to errors?

  • $\begingroup$ Loss could be monetary, but is far more general. You could think of it as something akin to utility, but it doesn't have to be actual utility, or even very much like it. It's just some specifies measure of the 'badness' of the outcome in some sense. $\endgroup$
    – Glen_b
    Oct 19, 2013 at 14:03

1 Answer 1


A loss function is a mathematical representation of anything bad or at least undesirable: the point is that it is therefore something you want to minimise.

Calling a loss function a cost is in general just terminology designed to be simple and evocative. The intention is to appeal to your sense that cost is something you want to avoid and (specifically and crucially) that (other things being equal) you prefer a smaller cost to a larger cost.

It doesn't necessarily imply a cost in any monetary, financial, economic or business sense.

Simple examples of a loss function arise when we consider the difference between some true or correct value $\theta$ and an estimate $\hat\theta$, which you would like to be as small as possible. Possible ways of taking that further are to work with $(\theta - \hat\theta)^2$ or $|\theta - \hat\theta|$, which are both loss functions. In either case there is a minimum loss of 0 when $\hat\theta = \theta$.

  • $\begingroup$ is it possible to have loss less than 0? $\endgroup$
    – ankc
    Oct 19, 2013 at 12:35
  • $\begingroup$ This is more a matter of convention than of logic. In practice, at least in my experience, loss functions are functions with zero or positive value. I don't think anything stops anyone calling something a loss function when that something might be negative. It's just like thinking that you want to minimise expenditure, but if you get some net income that is better yet than zero expenditure. $\endgroup$
    – Nick Cox
    Oct 19, 2013 at 12:45
  • 1
    $\begingroup$ @Tim not non-negative? The loss of loss functions that could be zero would be a real loss. $\endgroup$
    – Nick Cox
    Mar 17, 2015 at 11:49
  • $\begingroup$ @ankc Some definitions specify that loss function has to be non-negative to be the loss function (e.g. Robert, 2007, The Bayesian Choice) $\endgroup$
    – Tim
    Mar 17, 2015 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.