Is $-\sum_{n=1}^{N} \log(1+\exp(-t_ny_n)) $ the same loss as $\sum_{n=1}^{N}\{ t_n\log(y_n) + (1 - t_n)\log (1-y_n) \}$? I am trying to understand different forms of loss functions. I get confused with the terms cross entropy-loss and negative log-likelihood losses. I have seen the two following definitions of cross entropy-loss and negative log-likelihood losses (both terms are used for both formulas). The first one is (in the following paper: Improved Knowledge Graph Embedding using Background Taxonomic Information - Fatemi et al.: in the Objective Function and Training Section):

*

*$-\sum_{n=1}^{N}\{ t_n\log(y_n) + (1 - t_n)\log (1-y_n) \}$, where $t_n$ is the label (either 0 or 1) and $y_n$ is the probability

and the second one is (in the following paper: Low-Dimensional Hyperbolic Knowledge Graph Embeddings - Chami et al.: Equation 11):


*$\sum_{n=1}^{N} \log(1+\exp(-t_ny_n)) $, where $t_n$ is the label (BUT either -1 or 1) AND $y_n$ is NOT a probability but a similarity score (distance in this case)

I have changed the notation so it is the same for both formulas. Apparently as stated in the following post, cross entropy loss and negative log likelihood are equivalent. But are the above two formulas the same, I don't think so? Where is the difference? Is it because one time they use probabilities and the other time just a similarity score?
Thank you for your help! I really appreciate it! :D
(There are quite a few papers using those two formulas, the two above were just an example)
 A: Cross-entropies and likelihoods arise from specific probability models. When the probability model is the same, cross-entropy and likelihoods are different ways of writing down the same optimization problem.

*

*We can write down cross-entropy losses for lots of probability models besides Bernoulli probability models How to construct a cross-entropy loss for general regression targets?

*We can demonstrate the relationship between cross-entropy and the (log) likelihood for a Bernoulli model: the relationship between maximizing the likelihood and minimizing the cross-entropy
However, even if we are precise about our terminology, I think what you're asking is not about the terminology, but instead asking whether (1) and (2) refer to the same probability model, and if they do, how can one show that (1) and (2) are the same.
But there's a distinction which answers the question: (1) and (2) aren't the same probability model. (1) is used when $y_n$ is a predicted probability (of a Bernoulli trial), but in (2) $y_n$ is a distance, so they are not the same probability model. Distances are not probabilities because they can be arbitrarily large, but probabilities are at most 1. (The encoding of the targets is different between the two, 0/1 versus -1/+1, but this is not necessarily a deal-breaker; in certain problems, we can show equivalence between encoding choice.)
