From what I understand, the paired t-test is used when measuring the same "thing" in two different scenarios (e.g. one student's pre-test vs. post-test scores). However, is there a test that takes into consideration the variation in the measurment? Take the data set below. The objective is to determine if the two instruments produce similar results (Data 1 vs. Data 2). The testing is destructive so a direct paired t-test is not possible. However, each batch of material (15 in total) is tested in triplicate on each instrument. There is batch to batch variation, so I feel like doing a non-paired t-test loses that information. What is the best way to determine if the instruments are statistically indistinguishable?

Sample Data

  • 2
    $\begingroup$ A multilevel, or mixed, model is designed to solve this problem. $\endgroup$ Aug 8, 2018 at 19:37
  • 1
    $\begingroup$ Thanks! I am not familiar with a this model. Can you point me to where I can learn about it? $\endgroup$
    – KoldBeans
    Aug 8, 2018 at 20:25
  • 1
    $\begingroup$ I think I got it. I assume it is a multilevel regression, and if the coefficient for "instrument" is statistically significant, then the instrument is a factor that affects the measurement? $\endgroup$
    – KoldBeans
    Aug 8, 2018 at 20:29

1 Answer 1


You can solve this using a mixed model. You are measuring something (two different instruments, the treatments) on material that comes in batches. The idea is to divide each batch in two (roughly equal) parts, and using instrument A on one part, instrument B on the other.

A model can be written as $$ Y_{ijt} = \mu + b_i + \beta_t + \epsilon_{ijt} $$ where $i$ is batches, $j$ is material within batch, and $t$ (A or B) is instrument. $b_i$ is a random batch effect, $\beta_t$ is treatment effect. In R this can be done as (not tested)

mod  <-  lmer(Y ~ instrument + (1|batch), <etc>)  

This model is assuming constant variance of the error term $\epsilon_{ijt}$. This is a variance component model, see What is a variance component model?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.