One way anova or Paired-t test for same samples, different measurement technique

Question

I am measuring a sample thickness using two different methods in potentially different locations. We are using the same equipment (G R&R came back OK), but we gather 360 data points using 1 technique, and 120 data points using the other technique. The datapoints are likely collected in different locations around the part.

The sampling process is non-destructive (and therefore does not modify the sample) and our samples do not change over time. One sampling method is significantly quicker than the other. I would like to compare means, knowing that they will be slightly different, but we are okay if we are off slightly.

We have 73 different samples, each independent from one another. Would I use a paired t-test or one-way anova here (or would it be repeated measure ANOVA)?

I compared the data using a Pair-t test and it comes back with a P-value of 0.018 (reject the null which leads to the conclusion that the means are statistically different, but they are not significantly different).

Paired T-Test and CI: C2, C8

Paired T for C2 - C8

             N    Mean  StDev  SE Mean
C2          73  2809.1  126.6     14.8
C8          73  2802.5  126.3     14.8
Difference  73    6.63  23.38     2.74


99% CI for mean difference: (-0.61, 13.87)
T-Test of mean difference = 0 (vs ≠ 0): T-Value = 2.42  P-Value = 0.018

A one-way anova indicates that we fail to reject the null. However, I am under the impression that the samples can not be considered independent, because they are the same sample, but we could be measuring them in different locations. Would ANOVA be the wrong technique to use here? Or because we are measuring different locations on the same part, we should assume that the measurements are independent and therefore a paired-t test should NOT be used here.

http://www.statstutor.ac.uk/resources/uploaded/paired-t-test.pdf

Makes me think we should be using a paired t-test.

• A comparison of two different methods of measurement or two different treatments where the measurements/treatments are applied to the same subjects (e.g. blood pressure measurements using a stethoscope and a dynamap).

Answer 1

Since you collect the data from the same sample more than once, they are dependent.

I'm not really sure, what you need in the end, so I give you two suggestions:

• As far as I understand the issue, I would use an ANOVA with 1 between and 1 within subject factor. The between subject factor would be the type of method, the within subject factor the locations where you measured. I would only use data points which are measured by both methods. Otherwise you couldn't really compare differences in the measurement methods because your samples differ in thickness over different locations.
• I see a problem in comparing 360 data points with 120.You could build a mean for method 1 and for method 2 over all data points, and then compare the means with a paired t-test.

Answer 2

As the population of samples that are being collected are from the same object, you need to consider a paired t-test with dependent samples here. First and foremost, you need to check if normality assumptions hold true. One possible way to do that is to generate Q-Q plots and see how the data is distributed.

If the normality assumption does not hold true, then consider transforming the data (either $\log$ or $\exp$). Check again for normality using Q-Q plots. If normality fails, then look at non-parametric approaches such as Wilcoxon tests. They do not assume things about the distribution of the data.

A simple way to study these methods is by using G*Power 3.1 toolbox. They have a very simple user-interface that will allow you to select the type of test and generate a $p$-value.