2
$\begingroup$

In Elements of Statistical Learning II on page 349, the multinomial deviance loss function is given by $L(y,p(x))=-\sum_{k=1}^KI(y=G_k)f_k(x)+\log(\sum_{\ell=1}^Ke^{f_\ell(x)})$, but there is no explanation given as to why the index for the first summation is denoted $k$ and the one for the second $\ell$. What adds even more to my confusion is that on the previous page, they define the class probabilities $p_k$ as $p_k(x)=\frac{e^{f_k(x)}}{\sum_{l=1}^Ke^{f_l(x)}}$, using yet another index $l$ with no explanation. Does anybody here understand what the distinction is and what that means for the interpretation of the loss function?

$\endgroup$

2 Answers 2

0
$\begingroup$

It's done to prevent confusion, because the second summation is inside the first one. If you've two nested summations or for loops, you wouldn't use $i$ to index both, right?

Also, since the deviance loss is a function of $p_k(x)$, that $l$ comes from the expression of $p_k(x)$ you've written:

$$p_k(x)=\frac{e^{f_k(x)}}{\sum_{l=1}^K e^{f_l(x)}}$$

If you don't use another index $l$ for the summation, in the expression you'll confuse which $k$ to use (i.e. the subscripted $k$ int the formula, or in the summation index).

$\endgroup$
2
  • 1
    $\begingroup$ Thank you, that makes sense. Is there a reason that they use $l$ in the first equation and $\ell$ in the second equation, though? $\endgroup$
    – ballegard
    Mar 28, 2021 at 16:50
  • $\begingroup$ I don't think there is. Keen eye! They could've used completely another letter by the way, e.g. $m$. $\endgroup$
    – gunes
    Mar 28, 2021 at 17:14
0
$\begingroup$

The use of the index $l$ is just a 'dummy indexing variable' - it is used to avoid confusion with the fact that the indexing variable $k$ has already been used in the numerator.

You generally tend to see dummy indexing variables being used in machine learning settings particularly when you have a normalisation constant in context of discrete distributions in the denominator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.