# Quantifying 'survey bias' in reports

In situations where you know the individuals answering the survey are suspectible to some sort of bias within their responses, is there a way to simply integrate it into inferences made about the population? My initial thought is that some kind of adjusted confidence interval is the best place to do it.

A fictitous example:

I ask 100 individuals how much time they have spent on an activity in the past week (in minutes). I know from previous reports and research that when people are asked to re-collect time spent, they tend to under-estimate. On average, they under-estimate by about 20%.

Taking the responses direct from the 'survey' I get an average time of 100 minutes and a standard deviation of 25. From this I can create a standard confidence interval. But can I shift it and make it larger to account for the under-estimation?

A second fictitious example:

I ask 1000 individuals a sensitive question. 30% respond 'yes' and 70% respond 'no.' I know that those who answer 'yes' do not lie. However, of those who answer 'no', approximately 30% will lie.

I could take the responses directly from the survey and calculate a confidence interval for the percentage of individuals who would respond 'yes' in the population. But I want to incorporate the fact that some who answered 'no' would have lied. In which case, I almost have an asymmetric interval in which the true population parameter lies - it will have a wider interval above 30% than below.

P.S - I don't want to focus on how we can design surveys to avoid lying, bias etc.

• This doesn't show how to adjust a confidence interval but have a look at the Shy Tory Factor adjustment in UK political polling (also worth doing an internet search for that term). Note how polling firms make use of information from other questions and use it to reweigh responses (discounting the weight given to responses they deem less reliable), which is a little more sophisticated than what you propose. – Silverfish Jan 16 '15 at 12:29