I hope this is the correct place to ask this question. I'm currently doing research where I'm comparing two groups on their EEG frequency spectra. I wasn't too sure how to approach this.


I have the power values (dB) of multiple frequencies per subject, ranging from 0.6 to 30, in steps of 0.2, thus 148 data points.

enter image description here


In general, I would like to know where my two groups (red and black) differ on the power spectrum. I could run a t-test on every 148 bins, but I will surely run into several problems like multiple comparisons.


Which statistical test would be most appropriate here? Specific types of non-parametric tests? Is there some form of random testing that would be fitting here? Bootstrapping? How would that work? And would you in addition have any tips on how to implement this test (e.g. in R), even though that is out of the scope of this forum.

  • $\begingroup$ one option would be to use a regression model and identify the coefficient differences, so you need to find a sutable model for your power spectrum. $\endgroup$
    – seanv507
    Jul 10, 2018 at 14:27

1 Answer 1


I've came up with a solution myself, so maybe this helps someone in the future. A valid way to deal with this kind of data can be permutation testing (see here).

To quote from the article:

The nonparametric statistical test is performed in the following way:

(1) Collect the trials of the two experimental conditions in a single set.

(2) Randomly draw as many trials from this combined data set as there were trials in condition 1 and place those trials into subset 1. Place the remaining trials in subset 2. The result of this procedure is called a random partition.

(3) Calculate the test statistic on this random partition.

(4) Repeat steps 2 and 3 a large number of times and construct a histogram of the test statistics.

(5) From the test statistic that was actually observed and the histogram in step 4, calculate the proportion of random partitions that resulted in a larger test statistic than the observed one. This proportion is called the p-value.

(6) If the p-value is smaller than the critical alpha-level (typically, 0.05), then conclude that the data in the two experimental conditions are significantly different.

Based on a tutorial on youtube by Jonne Sälevä (2015), I created the following code, up to step 5 from above:

## Loop over all the columns of interest
for(i in range of columns){ 

  ## Split the data
  group1_data <- perm_data[perm_data$Group == "group1",]
  group2_data <- perm_data[perm_data$Group == "group2",]

  ## Make separate lists of group and dependent variable data
  combined_groups <- c(group1_data$Group, group2_data$Group)
  combined_x      <- c(group1_data$x, group2_data$x)

  ## Number of simulations 
  nsims <- 10000

  ## The observed difference in means     
  diff_obs <- mean(group1_data$x) - mean(group2_data$x)
  ## Initialize a variable to store the differences 
  diffs <- rep(NA, nsims)

  ## Permutation for loop
  for(k in 1:nsims){
    shuffled_labels <- sample(combined_groups, replace = FALSE)
    diffs[k] <- mean(combined_x[shuffled_labels == 1]) - mean(combined_x[shuffled_labels == 2])

  ## p-value = (number of more extreme differences than diff_obs/nsims)
  length(diffs[abs(diffs)>= abs(diff_obs)])/nsims
  perm_pvalue[index] <- mean(abs(diffs) > abs(diff_obs))

  • $\begingroup$ I agree that the optimal strategy here is a permutation test, especially since the power spectrum data is unlikely to be normally distributed (although a log transform of your data may get you closer). However there are two serious issues with above suggestion. The first is that you have a lot of dependencies in your data that you should taken into account when determining the significance (e.g. if neighbouring frequencies show the same pattern, this should strengthen your belief). The other is that the difference in means alone is unlikely to be a good indicator of difference. $\endgroup$
    – Mensen
    Oct 10, 2018 at 13:12
  • $\begingroup$ See ncbi.nlm.nih.gov/pubmed/?term=pernet+2015+tfce $\endgroup$
    – Mensen
    Oct 10, 2018 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.