A typical loss function in machine learning is:

$$L(\theta,x) = \mathcal L(\theta,x) +\sum_{\theta} |\theta|$$

I typically use the word “loss function” both for $L(\theta,x)$ and for $\mathcal L(\theta,x)$.

What terminology is there to distinguish these?

  • How do I refer to $L(\theta,x)$?

  • how do I refer to $\mathcal L(\theta,x) $?


1 Answer 1


I don't recall any specific name for the regularized loss function, people usually use this descriptive name. Loss function measures the error for a single sample, cost function measures it over all the data. The general name for the thing you optimize is called objective function, but if you want to be precise, better explicitly say that it is a regularized loss (or cost, they are often used interchangeably).

$$ \DeclareMathOperator*{\argmin}{arg\,min} \argmin_\theta \;\underbrace{\underbrace{\frac{1}{n} \sum_{i=1}^n \overbrace{\mathcal L(\theta,x_i)}^\text{loss function}}_\text{cost function} + \underbrace{\|\,\theta\,\|}_\text{regularization term}}_\text{objective function} $$

See Objective function, cost function, loss function: are they the same thing?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.