Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What is a good loss function for a predictive model used by gamblers?

I've been reading a bit about loss functions recently. I've always just went with MSE (e.g., for a couple of neural network projects) and didn't ask questions. I didn't realize exactly how arbitrary MSE actually is. And speaking of this, could somebody explain a simple practical situation where MSE can be derived as the "correct" loss function?

Anyway, I came across Log-Loss (which is the same as cross entropy) and am interested because I'm curious about some probabilistic models. I understand how one derives this loss function from information theory, but when we talk about probabilities, we're often involved in some form of gambling on an outcome. It's not clear to me that efficient transmission of information translates to the type of utility one is usually looking for in a predictive/probabilistic model.

If, for example, I had a model that was meant to predict the winner of the 2012 US presidential election, and I had used this to move money around on a future's market like Intrade, how might I determine a loss function for my prediction - assuming I'm able to continue making bets as the market fluctuates? Same type of thing should apply for any market where I am able to make handicapped bets on the occurrence of some event.

Or is this really regret, and is regret completely different than a loss function?

share|improve this question

I imagine it would depend on the payout scheme. For example if being wrong isn't a continuum but a binary either/or, then you would probably want to pick a loss function like mean absolute error or accuracy so that all incorrect answers are treated equally. On the other hand, if being overconfident -- saying something has a high probability, and then being wrong -- is exponentially bad, you would want to use log loss as a loss function, since log loss has an exponential penalty and an infinite penalty if you say something is 100% likely and you're wrong.

I imagine that if you sold derivatives / options (gambling in my opinion), you would want to use log loss since offering someone the option to buy stock ABC at 100.00 when it is currently at 80.00, and then seeing the price shoot up to 10,000.00 would be really bad and could bankrupt your company if sold in sufficiently large volumes. Getting wiped out (bankruptcy) would be considered infinitely bad.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.