0
$\begingroup$

I'm working with auction data, and I'd like to better understand how much the bid amount effects whether or not the bid is won. I have evidence to believe that the bid amount is not a reliable predictor, but I can't figure out how to quantify this.

A major hurdle in this specific case is that the win rate is extremely low, because it's likely that many of the auctions aren't actually "winnable", but are really just means for gauging the value of the items for sale. Logistic models I train (bid amount as predictor, with outcome 0 for loss and 1 for win) predict 0 for all validation examples. As such, I've been using log-loss to validate the model, and found that training on bid amount yields only a marginally better log-loss value than training on random normal variables. I'm looking for a single number that expresses whether or not the bid amount is actually predictive, so I know whether or not I should keep exploring this path or try another approach.

$\endgroup$
  • $\begingroup$ It seems like it's going to be quite hard to predict what bid is successful without some other measure of the value of what's bid on. In a sufficiently large sample with an auction house with a general range of values it seems like a random normal variable would roughly approximate that. $\endgroup$ – Bryan Krause May 17 '18 at 18:42
  • $\begingroup$ The items being bid on are services, and these services can take place anywhere in the country. Right now I'm partitioning the model by service type, but I don't have enough data to train models at the service-geo level $\endgroup$ – James Kelleher May 17 '18 at 19:38
1
$\begingroup$

You state that the logistic models predict 0 for all validation examples. I assume you are taking a classification perspective on logistic regression where you are rounding a probability less than 0.5 of winning down to 0 = "lost" and a probability above 0.5 up to 1 = "won". I suggest you examine the probability of wining as a function of item price.

I think you are interested in measuring the effect size of item price on probability of winning. In logistic regression it is common to look at the odds ratio. The odds ratio tells you the change in odds of success (e.g., winning) for a one unit change in the predictor (e.g., item price). You probably also want to account for the uncertainty in the estimate. Therefore, I suggest you calculate the 95% confidence interval for the odds ratio

$ (OR_{LL}, OR_{UL}) = exp\left(\hat{\beta_1} \pm z_{1-\frac{\alpha}{2}}s_{\hat{\beta_1}}\right) $

where $\hat{\beta_1}$ is the estimated coefficient of item price in your model.

$\endgroup$
  • $\begingroup$ possibly dumb question... can you explain what z_1, α, and s_βhat_1 are? $\endgroup$ – James Kelleher May 26 '18 at 0:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.