What's considered a good log loss? I'm trying to better understand log loss and how it works but one thing I can't seem to find is putting the log loss number into some sort of context. If my model has a log loss of 0.5, is that good? What's considered a good and bad score? How do these thresholds change?
 A: So this is actually more complicated than Firebugs response and it all depends on the inherent variation of the process you are trying to predict. 
When I say variation what I mean is 'if an event was to repeat under the exact same conditions, known and unknown, what's the probability that the same outcome will occur again'. 
A perfect predictor would have a loss, for probability P:
Loss = P ln P + (1-P) ln (1-P)
If you are trying to predict something where, at its worse, some events will be predicted with an outcome of 50/50, then by integrating and taking the average the average loss would be: L=0.5
If what you are trying to predict is a tad more repeatable the loss of a perfect model is lower. So for example, say with sufficient information a perfect model was able to predict an outcome of an event where across all possible events the worst it could say is 'this event will happen with 90% probability' then the average loss would be L=0.18.
There is also a difference if the distribution of probabilities is not uniform. 
So in answer to your question the answer is 'it depends on the nature of what you are trying to predict' 
A: Like any metric, a good metric is the one better that the "dumb", by-chance guess, if you would have to guess with no information on the observations. This is called the intercept-only model in statistics.
This "dumb"-guess depends on 2 factors :


*

*the number of classes

*the balance of classes : their prevalence in the observed dataset
In the case of the LogLoss metric, one usual "well-known" metric is to say that 0.693 is the non-informative value. This figure is obtained by predicting p = 0.5 for any class of a binary problem. This is valid only for balanced binary problems. Because when prevalence of one class is of 10%, then you will predict p =0.1 for that class, always. This will be your baseline of dumb, by-chance prediction, because predicting 0.5 will be dumber. 
I. Impact of the number of classes N on the dumb-logloss:
In the balanced case (every class has the same prevalence), when you predict p =  prevalence = 1 / N for every observation, the equation becomes simply : 
Logloss = -log(1 / N) 
log being Ln, neperian logarithm for those who use that convention.
In the binary case, N = 2 : Logloss = - log(1/2) = 0.693
So the dumb-Loglosses are the following : 

II. Impact of the prevalence of classes on the dumb-Logloss:
a. Binary classification case
In this case, we predict always p(i) = prevalence(i), and we obtain the following table :

So, when classes are very unbalanced (prevalence <2%), a logloss of 0.1 can actually be very bad ! Such as an accuracy of 98% would be bad in that case. So maybe Logloss would not be the best metric to use

b. Three-class case
"Dumb"-logloss depending on prevalence - three-class case :

We can see here the values of balanced binary and three-class cases (0.69 and 1.1).  
CONCLUSION
A logloss of 0.69 may be good in a multiclass problem, and very bad in a binary biased case.
Depending of your case, you would better compute yourself the baseline of the problem, to check the meaning of your prediction.
In the biased cases, I understand that logloss has the same problem as the accuracy and other loss functions : it provides only a global measurement of your performance. So you would better complement your understanding with metrics focused on the minority classes (recall and precision), or maybe not use logloss at all.
A: The logloss is simply $L(p_i)=-\log(p_i)$ where $p$ is simply the probability attributed to the real class.
So $L(p)=0$ is good, we attributed the probability $1$ to the right class, while $L(p)=+\infty$ is bad, because we attributed the probability $0$ to the actual class.
So, answering your question, $L(p)=0.5$ means, on average, you attributed to the right class the probability $p\approx0.61$ across samples.
Now, deciding if this is good enough is actually application-dependent, and so it's up to the argument.
A: I'd say the standard statistics answer is to compare to the intercept only model. (this handles the unbalanced classes mentioned in other answers)
cf mcFadden's pseudo r^2.
https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-what-are-pseudo-r-squareds/
Now the problem is what the maximum value is.  fundamentally the problem is that probability of an event is undefined outside a model for the events.  the way I would suggest is that you take your test data and aggregate it to a certain level, to get a probability estimate. then calculate the logloss of this estimate.
eg you are predicting click through rate based on (web_site, ad_id, consumer_id), then you aggregate clicks, impressions to eg web_site level and calculate the ctr on the test set for each web site.  then calculate log_loss on your test data_set using these test click through rates as predictions.  This is then the optimal logloss on your test set for a model only using website ids. 
The problem is we can make this loss as small as we like by just adding more features until each record is uniquely identified.
