In Elements of Statistical Learning II on page 349, the multinomial deviance loss function is given by $L(y,p(x))=-\sum_{k=1}^KI(y=G_k)f_k(x)+\log(\sum_{\ell=1}^Ke^{f_\ell(x)})$, but there is no explanation given as to why the index for the first summation is denoted $k$ and the one for the second $\ell$. What adds even more to my confusion is that on the previous page, they define the class probabilities $p_k$ as $p_k(x)=\frac{e^{f_k(x)}}{\sum_{l=1}^Ke^{f_l(x)}}$, using yet another index $l$ with no explanation. Does anybody here understand what the distinction is and what that means for the interpretation of the loss function?
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
It's done to prevent confusion, because the second summation is inside the first one. If you've two nested summations or for loops, you wouldn't use $i$ to index both, right?
Also, since the deviance loss is a function of $p_k(x)$, that $l$ comes from the expression of $p_k(x)$ you've written:
$$p_k(x)=\frac{e^{f_k(x)}}{\sum_{l=1}^K e^{f_l(x)}}$$
If you don't use another index $l$ for the summation, in the expression you'll confuse which $k$ to use (i.e. the subscripted $k$ int the formula, or in the summation index).
-
1$\begingroup$ Thank you, that makes sense. Is there a reason that they use $l$ in the first equation and $\ell$ in the second equation, though? $\endgroup$ Commented Mar 28, 2021 at 16:50
-
$\begingroup$ I don't think there is. Keen eye! They could've used completely another letter by the way, e.g. $m$. $\endgroup$– gunesCommented Mar 28, 2021 at 17:14
$\begingroup$
$\endgroup$
The use of the index $l$ is just a 'dummy indexing variable' - it is used to avoid confusion with the fact that the indexing variable $k$ has already been used in the numerator.
You generally tend to see dummy indexing variables being used in machine learning settings particularly when you have a normalisation constant in context of discrete distributions in the denominator.
