# Is this notation , $\ell(y,f(x;\theta))= -\log p(y|f(x;\theta))$, correct for the negative log probability loss function of a classifier?

In Murphy’s Probabilistic Machine Learning: An Introduction, he states that the loss function for a probabilistic classifier $$f(x;\theta)$$ is the following: $$\ell(y,f(x;\theta))= -\log p(y \mid f(x;\theta))$$ where $$x$$ is the input, $$y$$ the corresponding true output and $$\theta$$ the parameter vector of the model $$f$$.

My question is one of notation: Should it not rather be $$-\log p(y \mid f,x,\theta )$$? I feel that the original notation is saying the probability of the true label given the model output $$f(x;\theta)$$ taking a specific value $$f(x;\theta)$$ which doesn’t really capture what the “meaning” of this the probability distribution in this loss function (the probability of the true label $$y$$ of $$x$$, given the model choice $$f$$, the input $$x$$, and the parameters of the model).

Am I splitting hairs / missing something?

It doesn’t really matter which notation, though there’s a subtle reason you might prefer Murphy’s.

We could create an arbitrary variable $$u$$ of the correct type and measure the loss $$\ell(y, u)$$. Now Murphy’s version of the loss is $$- \log p(y \mid u)$$. Your proposed formula requires you to ‘remember’ how $$u$$ was created. That doesn’t make it wrong, but Murphy’s removes that assumption.

I did want to clear up one thing, though.

I feel that the original notation is saying the probability of the true label given the model output 𝑓(𝑥;𝜃) taking a specific value 𝑓(𝑥;𝜃) which doesn’t really capture what the “meaning” of this the probability distribution in this loss function

I don’t agree with the part that I italicized.

As a reminder, the probabilistic classifiers that Murphy describes here produce a probability distribution over classes, not a single ‘best’ class. For example, in the three-class iris dataset he describes, $$f(x; \theta)$$ is not $$\text{setosa}$$; it’s $$\{\text{virginica} \mapsto 0.3, \text{setosa} \mapsto 0.6, \text{versicolor} \mapsto 0.1\}$$, if you’ll forgive the notation.

That means that $$f(x; \theta)$$ has all of the information that you need to compute the probability of $$y$$ given $$f$$, $$\theta$$, and $$x$$. You no longer need access to the three ‘ingredients’. Instead, $$\ell$$ compares the ‘true’ label $$y$$ with the induced distribution $$u=f(x; \theta)$$.

For people who find this question in the future and are still curious, this definition of the loss function is Equation (1.8) of Murphy’s Probabilistic Machine Learning: An Introduction. It is on page 7 of the draft from March 8, 2021.

• Why does the my proposed formula require us to remember how $u$ was “created”? Apr 6, 2021 at 18:31
• It explicitly includes the ingredients $f$, $\theta$, and $x$ in the formula. $p$ is a function of four variables, not two. Apr 6, 2021 at 18:33