Experiment Setup

Each participant must perform volutary biceps flexions (each flexion is referred to as a trial $t_i$) while staring at a screen. EEG and EMG signals are recorded throughout the experiment.

There are two conditions (i.e., A and B), which determined what was shown on the screen during the trials. The number of trials per condition per participant ranges from 15 to 70.

Finally, there are two groups (i.e., G1 and G2) with 10 and 2 participants, respectively.

Graphically, the structure would be this. Note each participant experience both conditions (e.g., subject 1 from condition A is the same subject 1 from condition B).

enter image description here


I would like to test for the difference, in multiple dependent variables, between G1 and G2 given each condition. All dependent variables are numerical, but some are unbounded (e.g., event related desynchronization), some are strictly positive (e.g., power spectral density), and some are bounded (e.g., cortico-muscular coherence $\in$ [0, 1])

For some context, there are about 10 quantities that I would like to analyze (e.g., power spectral density, median frequency of the spectrum, EMG amplitude, peak readiness potential, peak event related desynchronization) for each condition. So, I would be testing approximately

10 dependent variables $*$ 2 conditions $=$ 20 group differences

Ideas/observarions so far

  • One common approach in the field is, for each quantity of interest and each condition, to average trials within a subject and then perform a test (e.g., t-test) between the two groups. However, I have been discouraged from doing so because of the low amount of participants in group G2 (i.e., 2).

  • Although I want to test for differences given a condition, I wonder if building a model that incorporates both conditions at the same time would be appropriate.

  • Maybe a Bayesian model would be desirable given how scarce data is. Under this framework, I had thought about a hierarchical linear model with trials as the observation unit, and a per group mean response.

I would appreciate any guidance about how to appropriately conduct the tests I want to perform given the low amount of data available, namely what model/framework to use, and why.

Also, as I need to back up any decision, pointers to papers and/or code are highly appreciated as well.

Thank you in advance!

EDIT: Clarified types of outcomes.

  • 2
    $\begingroup$ With only two subjects in one of the conditions your inferential analytical options are quite limited. I would suggest that you can only really use the study as a preliminary investigation and that means that the best things to do are to look at effective ways to show the data graphically. $\endgroup$ Sep 13, 2022 at 21:10

1 Answer 1


I'm sorry to say it, but this analysis is doomed before it starts.

You have only 2 participants in your smaller group, and 15-75 trials per condition per participant. Between-group differences in EEG signals are typically extremely small and noisy, and the chances of detecting a genuine effect in such limited data is essentially zero.

What's worse, since you're planning on looking at 10 different outcome variables, in two different conditions, there are lots (20) of opportunities for false positives to occur. Taken together, this means that any significant differences you do find are considerably more likely to be false positives than genuine effects (and if they do happen to be genuine effects, they'll be huge overestimates).

As a side note, between group effects are notoriously difficult to study using EEG. It's almost impossible to rule out the very strong possibility that any differences that do emerge reflect differences in motor artefacts, in electrode placement or signal quality, or even hairstyle. There are very few between group findings in the EEG literature that are believable. Luck's EEG textbook has a lot of good advice on good EEG designs for hypothesis testing. You may also want to look in power analysis for EEG designs.

Further Comments

  • Building a model that incorporates both conditions doesn't solve your problem here, since you want to compare the two groups and one of them has almost no data.

  • Similarly, Bayesian methods are nice, but they don't solve the problem. They provide a way of quantifying the uncertainty in your model, but they don't get around the fact that without more data you have no way of reducing that uncertainty.

  • $\begingroup$ "Building a model that incorporates both conditions doesn't solve your problem here, since you want to compare the two conditions and one of them has almost no data." Maybe you meant "since you want to compare the two GROUPS and one of them has almost no data."? $\endgroup$
    – Luisda
    Sep 19, 2022 at 19:43
  • $\begingroup$ Also, the data is already given, so thanks for the reference but designing a new experiment is simply not possible. Within was is possible, what would your recommendation be on how to proceed? In other words, if you had to do the analysis, what would you do? (Outside statistical testing as well) $\endgroup$
    – Luisda
    Sep 19, 2022 at 19:45
  • $\begingroup$ Yes, sorry, I did mean two groups. And I'm sorry to say it, but as far as I can see the data you have doesn't let you answer the question you're interested in (are there any differences between these groups?). $\endgroup$
    – Eoin
    Sep 20, 2022 at 9:23
  • 1
    $\begingroup$ Quoting John Tukey: "The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data." $\endgroup$
    – Eoin
    Sep 20, 2022 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.