1
$\begingroup$

I want to compare two tools, A and B.

I use tool A to measure 10 very different objects, making 5 repeated measurements for each object so I can record an accurate value.

I also do this with tool B.

Now I have some options:

  1. Average each 5 replicate measurements and do a single paired t-test. But there must be some error in each measurement set (5 replicate), so how do I account for this error?
  2. Do a separate t-test for each object. Let's say I can only reject the null hypothesis for some objects but not others. (And I don't know what aspect of the specific object affects the result.) How would I aggregate the result to get an overall comparison of A vs B?
  3. Do an ANOVA test. But in this case I only care about two categories, I don't know if ANOVA is appropriate.
Improve this question
$\endgroup$
3
  • $\begingroup$ Selection of an appropriate procedure depends in part on what you expect of the measurement dispersions: should they be approximately the same for every object? Or, perhaps, would you expect them to be proportional to each object's true value? $\endgroup$
    – whuber ♦
    Commented Apr 4, 2022 at 20:23
  • $\begingroup$ I expect the mean of the 5 replicates is very different for each object. Which procedure is then suitable for each case? a) the distribution of the 5 replicates is very similar, just around a different mean for each object, b) the distribution of the 5 replicates is different for each object due to random noise? $\endgroup$
    – jndi75
    Commented Apr 4, 2022 at 20:40
  • $\begingroup$ It sounds like you are trying to perform a gage R&R study. I suggest searching on that topic. $\endgroup$
    – Dave2e
    Commented Apr 4, 2022 at 23:03

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.