I would like to determine the measurement inaccuracy one of our laboratory machines in different conditions:

Let's say for simplicity's sake it is a b/w digital camera with 1024x768 pixels (786,432px). I have 32 pictures of the same test image. In theory these should be exactly the same. But I have varring conditions: time, lab assistant and room. The results vary and basically I would like to calculate the effect of the individual conditions.

My first question would be: What is a good similarity or error measure for the test pictures. I was thinking of the euclidean distance or 1-the peason correlation between the pixels of the two measurement points.

The second question is, how can I calculate the individual effect of the conditions on the measurement error. Unfortunately the test measurements were not systematically balanced (3 laboratory assistants in 4 rooms...)

In the end I would like to have a result like, the factor "room" has the smallest effect with 2% error rate followed by the laboratory assistant with 5% and so on.


1 Answer 1


First part of the question: For raw pixel comparison you can use the Mean Squared Error (MSE) or the Structural Similarity Index (SSIM). Because you have only 32 pictures to compare I would use the SSIM (more accurate, but slower). SSIM is implemented in Python via

from skimage.measure import structural_similarity as ssim
  • $\begingroup$ Thanks for your answer! The SSIM would be a super interesting approach. But I did not mention a problem: The true values of the test image are unknown. I just have repeated measures. My example ist simplified, in reality the lab machine is an optically working differential methylation analysis machine and the value of the pixels represents dna methylation status. $\endgroup$ Nov 23, 2019 at 16:16
  • $\begingroup$ Why as a starting point don't you perform SSIM to each couple of pictures and store the results in a symmetric matrix? Then you start analysing that. $\endgroup$ Nov 23, 2019 at 16:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.