I want to check the correctness of the data in a database with personal information like birthdate. So a survey is performed by interview and one of the questions was, "what is your date of birth". Comparing this answer with the information we have in our database will lead to an idea of the quality of the database. Out of a population of 1 mln people, lets say, 4.000 people are interviewed. Subtracting the two dates gives an idea of the error. Ideally is this zero for all the people in the survey. Which test can I use to say something about the significance of the error?

  • 1
    $\begingroup$ It depends on (a) how those people were selected and (b) your quality criteria for the database. For instance, if you need the dates to be correct you might draw one conclusion but if you only need them to be approximately right you might draw the opposite conclusion. What can you tell us about (a) and (b)? $\endgroup$ – whuber Sep 5 '17 at 20:24
  • 1
    $\begingroup$ The people were selected at random and the dates need to be correct, because they are used to calculate an insured sum. $\endgroup$ – andre de boer Sep 5 '17 at 20:27
  • $\begingroup$ You need to calculate a) the quality metric on a sample, then b) apply it to make a decision on the population. It's an inference problem. Your hypothesis would be that the quality metric on a population is within a limit that you set. $\endgroup$ – Aksakal Sep 5 '17 at 20:34
  • $\begingroup$ Thanks for your response, but how to calculate a quality metric? Is the difference between the date in the database and the date we get from the response already a quality metric? $\endgroup$ – andre de boer Sep 5 '17 at 20:39
  • 1
    $\begingroup$ For instance, you could ask yourself "is the mean or standard deviation of the difference between two birth date data elements is less than X?" Obviously there are differences. So, you estimate these metrics on a sample, then expand it to a population using ANOVA or similar approaches. $\endgroup$ – Aksakal Sep 5 '17 at 20:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.