# Intuitive explanation of logloss

In several kaggle competitions the scoring was based on "logloss". This relates to classification error.

Here is a technical answer but I am looking for an intuitive answer. I really liked the answers to this question about Mahalanobis distance, but PCA is not logloss.

I can use the value that my classification software puts out, but I don't really understand it. Why do we use it instead of true/false positive/negative rates? Can you help me so that I can explain this to my grandmother or a newbie in the field?

I also like and agree with the quote:

you do not really understand something unless you can explain it to your grandmother
-- Albert Einstein

I tried answering this on my own before posting here.

These are informative, and accurate. They are meant for a technical audience. They do not draw a simple picture, or give a simple and accessible examples. They are not written for my grandmother.

Logloss is the logarithm of the product of all probabilities. Suppose Alice predicted:

• with probability 0.2, John will kill Jack
• with probability 0.001, Mary will marry John
• with probability 0.01, Bill is a murderer.

It turned out that Mary did not marry John, Bill is not a murderer, but John killed Jack. The product of the probabilities, according to Alice, is 0.2*0.999*0.99=0.197802

Bob predicted:

• with probability 0.5, John will kill Jack
• with probability 0.5, Mary will marry John
• with probability 0.5, Bill is a murderer.

The product is 0.5*0.5*0.5=0.125.

Alice is better predictor than Bob.

• why does "product of all probabilities" work? This sounds like a relative of expectation maximization. – EngrStudent - Reinstate Monica Apr 20 '16 at 21:23
• Do you need a formal proof? It is in the "technical answer" mentioned by the topicstarter. Do you need an informal "grandmother" reason why? You say: suppose this fellow gave correct predictions. What is the probability that everything happen as it really happened? This is the product of probabilities. – user31264 Apr 20 '16 at 22:55
• "product of probabilities" isn't "grandma". log of product of probabilities is sum of log-probabilities, which they use in expectation maximization and call "expectation". I think it is also encoded in K-L divergence. ... I think in grandma-talk you could say" "most likely" = highest overall probability of multiple events. There are two get "highest": 1) maximize the combined probability or 2) minimize the negative combined probability. Most machine learning likes "gradient descent" or minimizing badness. Log-loss is the negative probability scaled by sample size, and it gets minimized. – EngrStudent - Reinstate Monica Jun 30 '17 at 13:11
• Here link they say "exp(-loss) is average probability of correct prediction." – EngrStudent - Reinstate Monica Jun 30 '17 at 13:16
• I liked the Bishop ref here. It is equation 4.108 and is the cross-entropy error function. – EngrStudent - Reinstate Monica Dec 8 '17 at 16:22