When conducting a permutation test, which variable should be shuffled?
Let's say I have two variables X and Y, and my test statistic is the correlation coefficient between the two variables. To test if the correlation is significant, I shuffle one of the two variables many times (e.g. 10000 permutations) and record how often the resulting correlation coefficient exceeds the original one. The proportion of exceeding correlation coefficients gives me a p-value that I can use for hypothesis testing.
So far so good, but when are there any restrictions on which variable I should shuffle? I guess it does not matter if I shuffle variable X or Y in the case of simple correlation coefficients. However, if I use the same procedure testing a multiple linear regression that has two independent variables, do I shuffle those or can I simply shuffle the dependent variable? If I need to shuffle the independent variables, do I need to shuffle them independent of each other (i.e. the observations of the two independent variables are now on different rows of the corresponding data-vectors)?
More specifically, this question originated from a problem where I correlate a variable that I derived from some raw data with another measured variable. Computing the derived variable from the raw data is expensive and therefore, I would rather shuffle the measured variable instead of the raw data. The alternative would be to shuffle the derived variable but I assume this would destroy statistical properties induced by processing the raw data (e.g. spatial filtering). I'd appreciate your thoughts on this.