I want to process automatically-segmented microscopy images to detect faulty images and/or faulty segmentations, as a part of a high-throughput imaging pipeline. There's a host of parameters that can be computed for each raw image and segmentation, and that become "extreme" when the image is defective. For example, a bubble in the image will result in anomalies such as an enormous size in one of the detected "cells", or an anomalously low cell count for the entire field. I am looking for an efficient way to detect these anomalous cases. Ideally, I would prefer a method that has the following properties (roughly in order of desirability):
does not require predefined absolute thresholds (although predefined percentages are OK);
does not require having all the data in memory, or even having seen all the data; it'd be OK for the method to be adaptive, and update its criteria as it sees more data; (obviously, with some small probability, anomalies may happen before the system has seen enough data, and will be missed, etc.)
is parallelizable: e.g. in a first round, many nodes working in parallel produce intermediate candidate anomalies, which then undergo one second round of selection after the first round is complete.
The anomalies I'm looking for are not subtle. They are the kind that are plainly obvious if one looks at a histogram of the data. But the volume of data in question, and the ultimate goal of performing this anomaly detection in real time as the images are being generated, precludes any solution that would require inspection of histograms by a human evaluator.
Thanks!