1
$\begingroup$

From what I understand, the paired t-test is used when measuring the same "thing" in two different scenarios (e.g. one student's pre-test vs. post-test scores). However, is there a test that takes into consideration the variation in the measurment? Take the data set below. The objective is to determine if the two instruments produce similar results (Data 1 vs. Data 2). The testing is destructive so a direct paired t-test is not possible. However, each batch of material (15 in total) is tested in triplicate on each instrument. There is batch to batch variation, so I feel like doing a non-paired t-test loses that information. What is the best way to determine if the instruments are statistically indistinguishable?

Sample Data

$\endgroup$
4
  • 2
    $\begingroup$ A multilevel, or mixed, model is designed to solve this problem. $\endgroup$ Commented Aug 8, 2018 at 19:37
  • 1
    $\begingroup$ Thanks! I am not familiar with a this model. Can you point me to where I can learn about it? $\endgroup$
    – KoldBeans
    Commented Aug 8, 2018 at 20:25
  • 1
    $\begingroup$ I think I got it. I assume it is a multilevel regression, and if the coefficient for "instrument" is statistically significant, then the instrument is a factor that affects the measurement? $\endgroup$
    – KoldBeans
    Commented Aug 8, 2018 at 20:29
  • $\begingroup$ Repeated measures ANOVA, or if normality and sphericity assumptions are not tenable, the Friedman test? $\endgroup$
    – Alexis
    Commented Jan 3 at 3:11

1 Answer 1

4
$\begingroup$

In experimental design your batches of material is called blocks. Then the blocks/batches are modeled as a random effect, and the A/B-instrument comparison is done within blocks.

You can solve this using a mixed model. You are measuring something (two different instruments, the treatments) on material that comes in batches. The idea is to divide each batch in two (roughly equal) parts, and using instrument A on one part, instrument B on the other.

A model can be written as $$ Y_{ijt} = \mu + b_i + \beta_t + \epsilon_{ijt} $$ where $i$ is batches, $j$ is material within batch, and $t$ (A or B) is instrument. $b_i$ is a random batch effect, $\beta_t$ is treatment effect. In R this can be done as (not tested)

library(lme4)
mod  <-  lmer(Y ~ instrument + (1 | batch), <etc>)  

This model is assuming constant variance of the error term $\epsilon_{ijt}$. This is a variance component model, see What is a variance component model?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.