Assume I have this kind of data:

$ A_{11}\quad A_{21} \\ A_{12}\quad A_{22} \\ A_{13}\quad A_{23} \\ B_{11}\quad B_{21} \\ B_{12}\quad B_{22} \\ B_{13}\quad B_{23} \\ C_{11}\quad C_{21} \\ C_{12}\quad C_{22} \\ C_{13}\quad C_{22} \\ ...\quad ... \\ $

where $A$, $B$, $C$, ... represent different individuals.

  1. For each individual $X$, I have measurements $X_{11}$, $X_{12}$, and $X_{13}$, which are three different measurements of a same physiological process P.
  2. For each individual $X$, I have also three other measurements $X_{21}$, $X_{22}$, $X_{23}$, which are again three different measurements of a same physiological process Q.
  3. The measurements ($X_{11}$, $X_{12}$, $X_{13}$) and ($X_{21}$, $X_{22}$, $X_{23}$) are measurements from two different physiological processes P and Q, that I would like to compare.
  4. The measurements are real numbers.

QUESTION: What kind of statistical test should I use to test whether there is a significant difference between the two physiological processes P and Q (The data for the first process is in the variables $X_{1i}$ and the data for the second process in variables $X_{2j}$, as explained above)?

My sample size is 10 individuals.

Does the answer change if I had more measurements (e.g. two more columns to the example data)?

  • $\begingroup$ This would be much more clear if you wrote it with indexing notation $X_{ij}$ $\endgroup$
    – Keith
    Commented Jul 30, 2014 at 7:46
  • $\begingroup$ I changed the notation now. $\endgroup$
    – user53078
    Commented Jul 30, 2014 at 7:51
  • 1
    $\begingroup$ You should say what questions you wish to answer from this data. $\endgroup$
    – Glen_b
    Commented Jul 30, 2014 at 7:54
  • $\begingroup$ I clarified the question now. I would like to know whether there is a significant difference in the two physiological processes. The data for the first process is in the first column and the data for the second process is in the second column. $\endgroup$
    – user53078
    Commented Jul 30, 2014 at 8:00

2 Answers 2


I am by no means an expert but I would suggest using a linear mixed effects model. The data could then for example be structured like this:

subj    process     measurement value
A       P           1           123
A       P           2           214
A       P           3           543
A       Q           1           234
A       Q           2           132
A       Q           3           674
B       P           1           952
B       P           2           348
B       P           3           233
B       Q           1           243
B       Q           2           940
B       Q           3           258
C       P           1           302

and so on. I just entered arbitrary numbers for the outcome "value" because I don't know what your outcome looks like. This assumes that the measurements for process P and Q are numerical and on the same scale.

In the mixed effects model you would specify value as the outcome/dependent variable, process as a fixed effect and both subject and measurement as a random effect with random intercepts. The maximum model (which you should probably use) would also contain random slopes for the process variable. If the order of measurements (1, 2, 3) doesn't matter, i.e. they were all taken in the same manner, using the same stimuli or whatever, you should leave it out.

In R, using lmer from the lme4 package, the code would look something like

model <- lmer(value ~ process + (process|subj) + (process|measurement))

or, without measurement as random factor:

model <- lmer(value ~ process + (process|subj))

However, summary(model) will not give you p values for the effects, only the parameter estimates. To assess significance you could use the mixed function from the afex package and just substitute mixed where I put lmer in the code example. The summary function will then give you a p value for the fixed effect.

As I said, this assumes that the outcomes are numerical. If they are categorical or ordinal, it gets a little more complicated.

I am a newbie in this as well so I welcome any corrections from more experienced users.


You have data $X_{ijk}$ where $i \in$[1,10] indexes your individuals, $j \in$[P,Q] indexes the types of processes and $k \in$ [1,3] indexes the measurements. Basically you want to collapse the $k$ dimension to get the result for each users. Then for each user you want to collapse the $i$ dimension to something you can use to compare $P$ and $Q$. The fist real question to answer is the question of error for each measurement. This is largely dependant on the type of sample/experiment. Is it a binomial (yes/no), multinomial(a,b,c,d) or measurement of a parameter taking a value on the real line?

In the parameter measurement case you would collapse $k$ by taking the mean of the three measurements then do a standard two sample t-test. This would change if the measurements were binomial, for example.

This is all the information I can give without knowing the kind of experiment/measurement.

And yes, the number of measurements always matters.

  • $\begingroup$ The measurements are real numbers. Unit is power. $\endgroup$
    – user53078
    Commented Jul 30, 2014 at 9:31
  • $\begingroup$ OK so you can then take a Gaussian assumption. Take the mean of your measurements to get a single value per user then do a two sample t-test (ie student's). I am sure people will comment about if you should weight by the error on your mean but this will get you started. Because you have so few measurements there could be cause for a lot of debate..... $\endgroup$
    – Keith
    Commented Jul 30, 2014 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.