I am overseeing the input of data from primary literature into a database. The data entry process is error prone, particularly because users must interpret experimental design, extract data from graphics and tables, and transform results to standardized units.
Data are input into a MySQL database through a web interface. Over 10k data points from > 20 variables, > 100 species, and > 500 citations have been included so far. I need to run checks of the quality of not only the variable data, but also the data contained in lookup tables, such as the species associated with each data point, the location of the study, etc.
Data entry is ongoing, so QA/QC will need to be run intermittently. The data have not yet been publicly released, but we are planning to release them in the next few months.
Currently, my QA/QC involves three steps:
- a second user checks each data point.
- visually inspect histogram each variable for outliers.
- users report questionable data after spurious results are obtained.
- Are there guidelines that I can use for developing a robust QA/QC procedure for this database?
- The first step is the most time consuming; is there anything that I can do to make this more efficient?