4
$\begingroup$

Years ago I read that there's a certain trick researches resort to when they want to obtain survey data regarding questions where one wouldn't expect people to answer honestly, such as "Do you beat your children?".

The trick had something to do with randomization and mixing the contentious questions with innocent questions and then doing some sort of analysis.

Unfortunately I can't remember what the method was called and how exactly it works. Does anyone have a pointer for me?

$\endgroup$
3
  • 5
    $\begingroup$ en.wikipedia.org/wiki/Randomized_response $\endgroup$
    – whuber
    Mar 14 '12 at 21:12
  • $\begingroup$ That sounds interesting. Can you elaborate a bit more on the protocol? $\endgroup$
    – Lagerbaer
    Mar 14 '12 at 21:26
  • $\begingroup$ @whuber I thought there were some issues around the accuracy of the method - did Andrew Gelman blog about this? $\endgroup$
    – Michelle
    Mar 14 '12 at 23:19
7
$\begingroup$

Another trick in addition to the Randomized Response method whuber mentioned is a case where the respondent can effectively conceal sensitive information in the form of an aggregate response. The researcher can then back out some measures of the sensitive information across the whole population, but not at the individual level. I can't remember what this is called either, but here's how you would do it:

For example, say you were trying to measure the incidence of erectile dysfunction. This is something many people would be reluctant to admit and a yes/no question would be ineffective. You could randomly include/not include ED in a list of, say, 7 other ailments, some common, some uncommon. You could then ask a question like "Out of the following list of 7 ailments, how many have you personally experienced within the last 90 days?" Having kept track of the respondents who were given the list with/without ED, you might find that people listed on average 3.2 ailments in the ED-less list, and 3.7 ailments in the ED-containing list. This would give an expected incidence of 50%.

You can see that one of the drawbacks is the error. If everybody had ED then the mean would be expected to be 1 higher in the longer list, and if nobody had ED the means would be expected to be identical. So you are measuring a difference of two means on the range of 0 to 1, so it is necessary to have a large sample size, so the confidence interval for each mean is much smaller than 1.

It would be important to carefully select the other ailments - you would not want hardly anybody to need to give a "1" for only the sensitive response, because then it may not feel as concealed for the respondent, and similarly, you would not want anybody to need to give a "7".

$\endgroup$
2
  • $\begingroup$ I think that was what I read about. Really neat :) $\endgroup$
    – Lagerbaer
    Mar 15 '12 at 15:36
  • $\begingroup$ It's called a 'List Experiment.' See (Blair, Imai, Lyall, 2014) for discussion of list experiments and endorsement experiments. There is also the 'Three Card Method.' See (Droitcour, et al., 2001). There's also one where you ask the respondent to flip the coin, and not show you the result, but answer truthfully if it's heads, and falsely if it's tails, but I don't remember the details. $\endgroup$
    – 5ayat
    May 16 '16 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.