How to deal with invalid data values such as with age (e.g., -99, 0, F1)?

Question

I have a data set that consists of 15 age values.

I want to clean the data before doing anything further. I have a few questions about data cleaning and data integrity.

What is the best treatment if the dataset contains an illegal value (like '${\rm F1}$', e.g.)? I thought taking the mean of the 14 remaining values and doing a mean substitution might be a good idea.

If there are values that cannot be right, such as $-99$ (no one can be $-99$ years old), is the best course of action a subjective treatment? Or should I use a statistical method, such as mean substitution? It seems logical to me to turn the $-99$ to a $+99$, and treat it as a data entry error.

Can $0$ be an age? The data set contains a $0$ value.

I am mostly looking to know if there is any formalised approach to dealing with these type of problems.

Answer 1

You should use common sense. Know how the data was recorded. Know the properties of the sample. Know any missing value codes. Know about the logical range of values both in general (e.g., age can't be less than zero) and for the application (if it's a study on adults, you shouldn't have age values less than around 16, 17, 18 or so). A few general categories of invalid data:

• Official missing data values: In some cases, a dataset will have an official set of values to indicate missing data. Common values include -9, -99, -999, NA, 99, etc. In these cases, you need to ensure that the software you use, is aware that such values are missing. In SPSS, there is a field to specify, missing data values. In R you can often specify this when reading data, or you can replace such values with NA.
• Invalid values that provide no information regarding true value: Some values are clearly invalid, and do not suggest what is the true value. This judgement is not intrinsic to the value. Instead it relates to the application. For example, to me "F1" as an age value has no meaning. But perhaps in your application it could mean something. In the case where the value has no meaning, the value recorded is equivalent to missing, so just replace value with missing.
• Invalid values that suggest true value: In other cases, the value will be invalid, but may suggest the true value. For example, you have a value of -43 and you are pretty sure that it is actually 43 and that the minus sign was a typo. Many of these cases take judgement and knowledge about what might have generated the error. Sometimes you can use additional information to inform your replacement of the hopefully correct value.

Whether a value of 0 or 99 is a legitimate value for age would depend on the nature of the dataset. You should know enough about the data to know whether it includes babies or the very old. Histograms of age may also be informative. If there are spikes at 0 and 99 with all the rest of the data between 18 and 60, this would suggest that the 0, and 99 data is invalid.

At the end of this process, your problem of invalid data will typically be transformed into one of missing data. At that point there are a wide range of issues related to assessment, deletion, imputation and so forth.