I have to perform outlier detection on population estimates for certain variables at the city level. For example, I might be estimating median income for a city and I want to know if there are any cities where the median income is an outlier with respect to the others.
My problem differs from a traditional outlier detection problem in two ways:
- If a city is an outlier, we won't be removing it from our analysis but rather we are just finding outlier cities in order to investigate why they are outliers.
- The "records" we are performing outlier detection on are estimates of population totals and not actual records themselves and as such our "records" are just point estimates that have a variance associated with them.
The question is, do we take this variance into account somehow? For instance, if most of our cities had a median income around 50k but one had a median income of 500k we would say the 500k city was an outlier. But what if that 500k city had a variance so large that a 95% CI covered 50k? Is it no longer an outlier? What if all of our cities have giant CIs. Is it possible to determine outliers from the point estimates alone?
One thought I had would be some sort of simulation where we sample estimates from the CIs of each city and perform many outlier tests and then analyse those results.