Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

Let's say I collect data from an experiment where for each measurement device, 5 trials are conducted at different temperatures. Each device is calibrated separately, and the 5 trial temperatures are constant across devices. In other words, I have values at Temperature 1, 2, 3, 4, and 5 from each device.

What is an appropriate test to see whether the values from a given measurement device, adjusted for calibration, are significantly different from the other measurement devices? My guess is that the test must be conducted on the differences between the trial values to compensate for the different calibrations.

share|improve this question
Could you please clarify: are there three or four variables? i.e. device + temperature + measurement, or device + calibration + temperature + measurement. – Jason O. Jensen Apr 29 '12 at 6:32
@JasonO.Jensen: Each device is measuring from the same input but is calibrated differently. For instance, at T1, given that x is the true value, device 1 might report a measurement of x while device 2 reports x+5, so at T2, with y as the true value, device 1 might report y and device 2 might report y+5. Device 2 is thus calibrated by +5 from the true value, but is accurate when adjusted for calibration (i.e. T2-T1 is approximately equal for both devices). I'm trying to test whether the adjusted measurements of some device 3 are significantly similar to those of the other devices. – Maxim Zaslavsky Apr 29 '12 at 8:41

1 Answer

up vote 1 down vote accepted

Assuming the device measurement error is the same magnitude across temperature levels and that you can easily factor out calibration differences you can do pairwise comparisons between the means measured with the different devices. I.e. the calibration-adjusted mean across all five measurements for device 1 and the calibration-adjusted mean across all five measurements for device 2.

As doing multiple pairwise comparisons will reduce the overall certainty of the results if the same significance level is used in all of them it is best to use Tukey's Honestly Significant Difference test to perform the comparisons. This test can control for an arbitrary number of pairwise comparisons.

That said, if you really want to know whether the different devices produce different results, it would be best if you had more than 5 data points per device.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.