Statistical tests to do data quality checks

I want to identify bad responses in the data we get from our vendors. To assess the quality of the data, what statistical tests can i apply to identify bad respondents?

I performed Discriminant analysis to identify the % similarity of responses among surveys completed by a particular interviewer. How can i further authenticate my findings?

What i mean by bad responses is: Interviewers who have very high similarity in the responses they collect in the survey. they might tick for "4" for a particular question for all the respondents (Flat Liners or Speeders)

Suppose an interviewer has conducted 10 surveys and i notice similarity in the answers of those questionnaires for each question. Is there any statistical test which authenticates my finding that the interviewer has probably filled responses himself or has ticked the same response for each question? How can we check a particular data using Statistical tests? I hope i have clarified my question.

-
You need to tell us more about what you mean by "bad responses" and "bad respondents"; you also seem interested in "bad interviewers". –  Peter Flom Feb 9 '12 at 11:43
What i mean by bad responses is: Interviewers who have very high similarity in the responses they collect in the survey. they might tick for "4" for a particular question for all the respondents (Flat Liners or Speeders) –  Shantanu Feb 9 '12 at 12:22
Suppose an interviewer has conducted 10 surveys and i notice similarity in the answers of those questionnaires for each question. Is there any statistical test which authenticates my finding that the interviewer has probably filled responses himself or has ticked the same response for each question??? How can we check a particular data using Statistical tests?? I hope i have clarified my question.. PLEASE HELP... –  Shantanu Feb 9 '12 at 15:30
My first thought would have been to look at methods to detect cheating on multiple choice items, although methods to handle extreme response style or inter-rater analysis are likely candidates. –  chl Feb 9 '12 at 18:56
I'm not sure why you need a statistical test. If you suspect that the interviewer has ticked the same response for the same question across all interviews, then the sum or mean of responses for that interview can be examined. That will tell you absolutely that the same answer is chosen every time. Falsifying survey data requires another, trusted interviewer to interview the same respondent, so you have two results to compare. –  Michelle Feb 9 '12 at 19:22
show 1 more comment

The keywords to search for are "interviewer falsification". AAPOR/SRMS guidelines are a good starting place, and RTI system is useful, too. As far as I know the evidence, the interviewers may be able to get the first moments OK, but they have a more difficult time with higher order moments, so falsified data may be detected via unusual variances and correlations/crosstabs.

-
(+1) That looks indeed like the right keyword to search for! –  chl Feb 10 '12 at 12:50
Hi...thanks for your suggestions. I am using discriminant analysis to group the responses that a particular interviewer gets for a question which basically tells me the % of similarity of the groupings. But this is countered by saying that the reason you are having similar responses is because your questions are similar. Now how do i prove that my questions are not similar. Is there any statistical procedure for this...thanks a lot all of you for your help... –  Shantanu Feb 10 '12 at 14:43
In psychometrics, somewhat similar ideas are covered under the umbrella of "divergent validity": if items are measuring something different, they should vary. That's again the second order moment issue of (co)variance between items. I am not even sure I see how you apply discriminant analysis here. –  StasK Feb 10 '12 at 19:48

I don't know of a statistical test that does what you want, although there may be one. Probably, though, you will need to define "bad interviewer" more precisely.

There is some literature on interviewer effects, and that might be a place to start looking.

-