A typical loss function in machine learning is:
$$L(\theta,x) = \mathcal L(\theta,x) +\sum_{\theta} |\theta|$$
I typically use the word “loss function” both for $L(\theta,x)$ and for $\mathcal L(\theta,x)$.
What terminology is there to distinguish these?
How do I refer to $L(\theta,x)$?
how do I refer to $\mathcal L(\theta,x) $?
I don't recall any specific name for the regularized loss function, people usually use this descriptive name. Loss function measures the error for a single sample, cost function measures it over all the data. The general name for the thing you optimize is called objective function, but if you want to be precise, better explicitly say that it is a regularized loss (or cost, they are often used interchangeably).
$$ \DeclareMathOperator*{\argmin}{arg\,min} \argmin_\theta \;\underbrace{\underbrace{\frac{1}{n} \sum_{i=1}^n \overbrace{\mathcal L(\theta,x_i)}^\text{loss function}}_\text{cost function} + \underbrace{\|\,\theta\,\|}_\text{regularization term}}_\text{objective function} $$
See Objective function, cost function, loss function: are they the same thing?
