I am analyzing a set of costs for contractor services for a particular city. These data have been extremely difficult to collect due the difficulty of contacting contractors and reluctance of contractors to divulge their pricing information. Factors are "service type" and "contractor".

We generally have quotes from more than one contractor on each service and a number of contractors perform most of the services. For a few services we have fairly good replication (i.e. ~15 quotes), for some of the services we have some degree of replication (i.e. 5-10 quotes per service). But for about a third of the services replication is very poor, i.e. <5 quotes per service.

The data have proven extremely challenging and time consuming to collect and I doubt if there will be more.

The upside to this story is that there are several construction cost estimation tools that can give us estimates of the costs of these services. And while these costs wouldn't be specific to the area we're interested in, we think we can assume that the prices of the services relative to each other would be similar to what we expect to see in the area we're looking at.

Sometimes elegance is the first casualty of pragmatism, but this is what I'm thinking of doing-- I am planning to collect a parallel set of cost estimates from these cost estimation tools and compare the relationships between the results of our surveys to the relationships of costs within this set of parallel cost data.

Essentially using the better replicated data points in our dataset as anchors and generating expectations for our poorly replicated data points based on the relationships from the parallel dataset.

I'm not even sure what this approach would be called and it feels like a very specific type of issue, so it's difficult to research online. But I'm wondering if there's any theoretical basis for this type of approach that's been worked out. Shall I buy a book on Bayesian stats?

Most of my experience in grad school was analyzing data from designed experiments, so this sort of situation is interesting, but definitely foreign to me.

Thanks in advance for your help.

  • $\begingroup$ It is unclear to me what the actual problem is that you are attempting to solve by collecting this parallel dataset, and how exactly this parallel dataset will solve that problem for you. Right now I read all of this as saying something like "I think my dataset is not great and I think more data would be better." It's hard to know how to respond to this. Maybe you can explain more clearly, directly, and precisely what you are trying to accomplish. $\endgroup$ – Jake Westfall Nov 4 '13 at 2:38
  • $\begingroup$ Would multilevel regression with post-stratification (Mister-P) help? andrewgelman.com/2013/10/09/mister-p-whats-its-secret-sauce $\endgroup$ – Jeremy Miles Jan 4 '14 at 1:35
  • $\begingroup$ Maybe look into multiple imputation and how to cross-validate it. I'd give a more complete answer myself but it would just entail me looking up the same. $\endgroup$ – shadowtalker Jan 4 '14 at 10:42

This sounds more like a data collection problem. If you have smaller sample sizes there is nothing you can do from a statistical perspective to extract more information.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.