3
$\begingroup$

I am trying to compare readings from an old flowmeter and a new digital flowmeter to find out if they are significantly different. With the new meter I did readings under different conditions eg. calibrated at 30 seconds, calibrated at 60 seconds, uncalibrated at 30 seconds and uncalibrated at 60 seconds. I want to compare my old flowmeter (first column on dataset) to the other readings (columns 2:5 on dataset). I used anova to compare the groups and found some differences but I am stuck with TukeyHSD...My question is: Is this the correct way to do the comparisons...I would appreciate some advice on how to accomplish this task...Here is my dataset and code....

    meter <- structure(list(SiteNumber = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L), .Label = c("site1", 
"site2", "site3", "site4"), class = "factor"), old_dev = c(2.09, 
1.927, 2.11, 2.114, 2.11, 1.995, 2.605, 2.563, 2.393, 2.286, 
2.529, 2.371, 2.214, 2.128, 2.302, 2.204, 2.274, 3.513, 2.61, 
3.502, 3.155, 3.131, 3.55, 3.656, 3.666, 3.618, 3.023, 3.226, 
3.649, 3.519, 3.322, 3.352, 3.323, 3.144, 2.869, 3.383, 3.076, 
2.978, 3.189, 2.934, 2.813, 3.015, 3.014, 3.056, 2.952, 2.988, 
2.976, 3.049, 2.869, 2.965, 2.895, 2.064, 1.333, 1.432, 1.599, 
1.52, 1.51, 2.734, 2.182, 2.71, 2.488, 2.242, 1.998, 1.175, 1.791, 
2.152, 3.414, 3.278), new_dev_unc30 = c(2.6, 2.6, 2.7, 2.6, 2.3, 
2.6, 3.2, 2.9, 2.1, 2.8, 1.8, 2.6, 2.5, 1.4, 2.7, 2.6, 2.2, 4.1, 
3.4, 3.7, 4, 4, 4.5, 4.5, 1.7, 2.9, 2.4, 2.9, 3.7, 3.3, 4, 1.7, 
3.5, 1.9, 3.3, 4.1, 3.7, 4, 3.6, 2, 2.4, 3.7, 2, 2.8, 3, 1.8, 
1.8, 3, 1.9, 3.1, 1.1, 2.2, 1.6, 1.2, 1.4, 1.3, 1.4, 2, 1.9, 
1.4, 0.7, 2.1, 1.2, 1.8, 1.3, 1.2, 0.8, 2.5), new_dev_unc60 = c(2.7, 
2.2, 2.6, 2.3, 2.7, 2.4, 3.1, 2.8, 2.3, 2.9, 1.8, 2.4, 2.6, 1.3, 
2.7, 2.6, 2.2, 4.1, 3.8, 3.9, 4.5, 4, 4.2, 4.6, 2, 2.6, 2.6, 
3, 3.4, 3.5, 3.9, 2, 3.4, 2.3, 3.3, 4.2, 3.7, 3.9, 3.5, 2.1, 
2.4, 3.1, 2.1, 3, 3.2, 1.9, 1.9, 3, 2.1, 3.1, 1.3, 1.5, 1.6, 
1.2, 1.8, 1.1, 1.2, 2.1, 2, 1.4, 0.8, 1.9, 1.1, 0.9, 1.4, 1.5, 
0.8, 2), new_dev_cal30 = c(2.4, 2.5, 2.5, 2.8, 2.4, 2.3, 3.1, 
2.6, 2, 3.5, 1.6, 2.5, 2.5, 1.7, 2.9, 2.5, 2.4, 4.5, 3.3, 4.3, 
4.3, 4.3, 4.4, 4.5, 1.1, 1.9, 4, 3.5, 3.1, 3.8, 3.5, 2.2, 3.5, 
2, 3.6, 3.7, 4, 3.5, 3.8, 2.4, 2.7, 3.3, 2.7, 2.3, 3.2, 2.3, 
0.9, 3.2, 2.9, 3.1, 0.9, 2.3, 1.3, 1.4, 1.5, 1.1, 1.3, 2.7, 1.7, 
2.1, 0.7, 1.9, 1.2, 1, 1.6, 1.1, 1.4, 1.6), new_dev_cal60 = c(2.6, 
2.5, 2.4, 2.3, 2.5, 2.3, 3.2, 2.8, 1.9, 3.1, 2.1, 2.7, 2.4, 1.8, 
2.7, 2.6, 2.3, 4.3, 3.5, 4.3, 4.3, 4.3, 3.8, 4.4, 1.4, 2, 4.2, 
3.3, 3.2, 3.3, 3.4, 2.1, 3.4, 2.5, 3.8, 3.8, 3.9, 3.6, 3.5, 2.4, 
2.9, 3.1, 2.3, 2.6, 3.1, 2.2, 0.8, 3.2, 2.6, 3.1, 0.9, 1.7, 1.2, 
1.5, 1.2, 1, 1.1, 2.6, 1.6, 1.9, 0.6, 2.3, 1.4, 1.2, 1.5, 1.2, 
1.4, 1.4)), .Names = c("SiteNumber", "old_dev", "new_dev_unc30", 
"new_dev_unc60", "new_dev_cal30", "new_dev_cal60"), class = "data.frame", row.names = c(NA, 
-68L))

Here is what I did:

str(meter)
attach(meter)
boxplot(list(old_dev,new_dev_unc30,new_dev_unc60,new_dev_cal30,new_dev_cal60))
summary(firsttest <- aov(old_dev ~ new_dev_unc30 + new_dev_unc60 + new_dev_cal30 + new_dev_cal60 ))
plot(firsttest)
TukeyHSD(??????)
|cite|improve this question
$\endgroup$
  • $\begingroup$ since I am trying to compare the old_device to the new device calibrated and uncalibrated to see if there are significant differences, is it OK to use aov like this: summary(test1 <- aov(old_dev ~ new_dev_unc30)); summary(test2 <- aov(old_dev ~new_dev_unc60)); summary(test3 <- aov(old_dev ~ new_dev_cal30)); summary(test4 <- aov(old_dev ~ new_dev_cal60)) to compare the old device with the different modes of the new one? $\endgroup$ – user9093 Feb 9 '12 at 22:50
  • $\begingroup$ NO! The syntax for formulas with aov is always dependent.variable ~ independent.variable where independent.variable needs to be a nominal factor (see ?factor). And please edit your question and do not post questions as answers. $\endgroup$ – Henrik Feb 10 '12 at 9:59
2
$\begingroup$

The analysis you are doing seems to be incorrect (as I see it). Additionally, from you description I do not understand your data. Your data is a two-factorial design with the factors SiteNumber (SiteNumber1 to SiteNumber4) and the device (OldDevice and 4 times the new device). This does not correspond with your description.

In the following I submit the data to a 2-way ANOVA (using the reshape and ez package) with both factors (variale = device). It reveals only a main effect of SiteNumber but no effect of the device:

library(reshape)
library(ez)

meter.long <- melt(meter)
meter.long$id <- factor(seq_len(dim(meter.long)[1]))
ezANOVA(meter.long, wid = .(id), dv = .(value), between = .(SiteNumber, variable))

gives:

$ANOVA
               Effect DFn DFd           F            p p<.05         ges
1          SiteNumber   3 320 107.9167057 2.734194e-48     * 0.502912712
2            variable   4 320   0.6228809 6.464967e-01       0.007725858
3 SiteNumber:variable  12 320   1.0357982 4.153921e-01       0.037390111

$`Levene's Test for Homogeneity of Variance`
  DFn DFd      SSn     SSd       F            p p<.05
1  19 320 16.15142 58.6906 4.63488 2.008842e-09     *

In sum this analysis allows one conclusion regarding your question: There is no evidence that there is a difference between measures from the two devices independent of the calibration.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks Henrik, Would it make any difference if I remove the siteNumber column? I only care about comparing the old device (old_dev) with all the other columns (old_dev,new_dev_unc30,new_dev_unc60,new_dev_cal30,new_dev_cal60). The siteNumber doesnt matter $\endgroup$ – user9069 Feb 9 '12 at 3:00
  • $\begingroup$ Leaving Site Number out shouldn't change anything. You can try it out by removing SiteNumber from the call to ezANOVA. $\endgroup$ – Henrik Feb 9 '12 at 7:21
  • $\begingroup$ Is there another way I can do anova besides the ez package? $\endgroup$ – user9093 Feb 9 '12 at 21:44
  • $\begingroup$ yes. ez is simply a wrapper of the Anova function in the car package (which I also recommend). You could also use lm or aov from base R, but it is less comfortable (and does not support type 3 SS). I would go with ez or car. $\endgroup$ – Henrik Feb 9 '12 at 21:59
1
$\begingroup$

I would plot the values and perform a least products regression to see if there is anything interesting going on in the data, but there do seem to be statistical tests that would be appropriate. Comparing methods of measurement is an interest of my friend John Ludbrook. He has a review of statistical methods that you can see here: https://www.ncbi.nlm.nih.gov/pubmed/12060093

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.