Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm investigating optimal bidding in auctions, and am using logistic regression to predict the probability of winning an auction given a set of explanatory variables (e.g. the price I bid, number of competing bids etc).

One explanatory variable I want to use is the second highest price that was paid. However, by the design of the auction, I only observe the second highest price paid when I am the highest bidder (i.e. when I win the auction).

This missing data is a major issue as my dataset indicates that there is a winning bid only ~20% of the time, hence I don't know the second highest price paid 80% of the time. Yet intuitively, I don't want to drop this variable as it seems to me knowledge of the second highest bid is extremely valuable in determining my chances of being the winning bid.

Thus are there any standard methods to cope with this kind of missing data for logistic regression?

share|improve this question
    
This sounds to me more like a missing data problem than a censoring problem. –  Peter Flom Jul 21 at 10:38
    
@PeterFlom updated post to reflect your comment –  Milo Chen Jul 21 at 10:40
    
This is definitely a censoring situation. Considering it as such is more fruitful than treating it as missing data because the mechanism of the censoring is clear and leads directly to useful likelihood-based models and estimation. (cc @Peter) –  whuber Jul 21 at 13:53

2 Answers 2

up vote 2 down vote accepted

I am afraid you cannot expect to find some "canned" solution to your problem. Most methods for handling missing data assumes "missing at random" or even "missing completely at random" (you can gogle those terms!). Your problem seems definitely to be a problem of informative missingness. Then you will need to model the mechanism of missingness, and maybe model the "second highest bid" as a response, given some covariables (which might include the winning bid).

From there you can try to build a custom model. You can google for "informative missingness" to get some ideas.

share|improve this answer

@Kjetil gave a good answer.

One possible simple alternative, if you have enough auctions, is to run two models: One with the data that has both highest and second highest and one that has just the highest.

An advantage of this approach would be that each model will be considerably simpler than a full model with both. But a disadvantage is that you won't be able to use the 2nd highest bid at all unless you actually have it.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.