# Tagged Questions

A permutation is a rearrangement of values, items or objects, which arise in various statistical contexts. Permutation tests are statistical tests based on all possible rearrangements of data that are consistent with the null hypothesis.

195 views

### Permutation test for the significance of several dependent regression coefficients from separate models

I have $i>100$ Cox regression models which differ only in one explanatory variable, $x_i$. They have the same outcome data and the same other explanatory (adjustment) variables. I want to control ...
124 views

### Controlling type I error in permutation tests

I'm implementing a permutation test for a test statistic with unknown distribution. Apart from my concrete situation (see below), I'd like to know first If and under what circumstances permutation ...
126 views

### Permutation paired test with signed statistic

I have one paired data from 7 patients: $$(X_i ,Y_i)=\{(5,1),(-2,-4),(4,2),(-6,-3),(-3,-3),(-5,2),(3,1)\},\quad (i=1,\ldots, 7)$$ where $X$ means placebo $Y$ means the test drug. My classmate asked ...
164 views

### How many random permutations to cover all possible permutations?

I have code that generates a random permutation. In my case, a permutation consists of N binary features, and each of the N features is set or unset randomly. How many times must I generate a random ...
214 views

### Permutation test to test significance of skewness/kurtosis of two distributions?

To test the significance of the skewness difference between two distributions with $N_1$ and $N_2$ samples, would the following test work: Create a single array of all the samples from both ...
240 views

### lmPerm p-values and multiple testing

I've started using lmPerm in order to perform regressions in R. The equation I want to fit has the form: ...
280 views

### What is this permutation?

I was recently looking for ways to resample time series, in ways that Approximately preserve the auto-correlation of long memory processes. Preserve the domain of the observations (for instance a ...
89 views

### Citation for permutation test

I am using a permutation test to verify the statistical significance of an improvement in multi-class classification performance. What citation would be appropriate? Would it be fine to cite ...
128 views

### Limitation on group size with non-parametric permutation tests

I would like to know if there is any limitation in the size of the groups when you are implementing non-parametric permutation tests. My case belongs to the realm of medical imaging. I am comparing ...
473 views

### Two-sided permutation test vs. two one-sided

I am working with a set of genes for which I have both methylation values $\beta$ (continuous on unit interval) and gene expression $E$ (non-negative continuous) that I want to test for correlation ...
188 views

### Correlation analysis and correcting p-values for multiple testing II

I am currently facing problems dealing with lots of individual simple correlations (28 individual DTI connections strengths, individual behavioural scores on 31 time points). That would be 28x31 ...
65 views

### Unique subsequences of the human genome

I am working on a project involving genomes. I am going to divide a genome into subsequences, and then do something with those subsequences. I want to know if the subsequences that I produce are going ...
101 views

### Analyzing ratios or numerator/denominator, with weights, measured over time at irregular intervals

I have gathered information on the rate increases allowed by a government regulator over about 17 years. As background, utilities in this area must apply to the regulator if they wish to raise their ...
112 views

### Monte Carlo test for comparing curvature of binomial response surfaces from effective degrees of freedom of GCV-fitted splines

I would like to compare the curvature of two response surfaces, each of the form: binomial ~ continuous variables1-5 I think it would be appropriate to use the effective degrees of freedom of a ...
704 views

### Permutation tests in R for correlations

What's the easiest way to run permutation tests for a bivariate Pearson correlation matrix in R? I have tried running MPT.Corr, Multivariate Permutation Test for Correlations by Urbano Blackford et ...
211 views

### Doubling the tails in two-sample permutation test

Suppose we have two samples and we wish to determine if they are drawn from the same distribution, the samples A,B being composed of some integers say. If we test this using a two-sample permutation ...
147 views

### To what degree is FWE correction required for multiple permutation based t-tests

I am performing multiple t-tests to compare conditions of data. I have had to resort to permutation based t-tests (based on difference of means) due to high variance between samples. My question is ...
1k views

### How to resample in R without repeating permutations?

In R, if I set.seed(), and then use the sample function to randomize a list, can I guarantee I won't generate the same permutation? ie... ...
789 views

### T-test vs. Permutation test on (not surely) normal data

This question follows my previous one. I will now provide my data in order to help you help me :) As previously said, I have N=42 from a controlled experiment. People made a questionnaire assessing ...
311 views

### Restricted permutations

I have some multivariate data and want to investigate the effect of some environmental gradient. I want to use capscale but I donĀ“t know how to deal with the permutation scheme. I have made up some ...
238 views

### How to define the rejection region for a permutation test?

I am trying to find out the rejection region for a permutation test about the difference between medians where: ...
736 views

### Is there a way to compare linear regression slopes by permutation tests?

I have some species $\times$ year matrices and analyse the temporal dynamics by regressing distance between years on $\sqrt{ {\rm time \ lag} }$. Since the data points are not independent and since I ...
632 views

### Generating all possible unique combinations of dataset with two variables

I am fairly new to R and I have been reading all related topics but still I don't manage to solve my rather simple problem. I have a dataset with 5 cases for two variables (X,Y), for instance: ...
271 views

### NAs in summary of linear model generated from significant genes for continuous response variable identified using SAM

I have a gene expression data-set with log2-transformed expression values (no NAs) for 495 genes for 59 samples for which values of a continuous response variable (r) are also known (no NAs). I want ...
2k views

### How to generate nD point process?

Context: Poisson point processes (PPP) are widely discussed in the literature. In the following figure a framework to generate two-dimensional PPP is demonstrated. First the area being studied (part ...
681 views

### Bootstrap and randomization tests to compare paired data sets

I have 50 values set as "true" in a simulation and attempts to recover these "true" parameters using 2 different models. Here's 3 (instead of 50) runs just to show the data layout (these are made up ...
908 views

### False discovery rate from permutation testing?

I have conducted a search for genetic interactions using a simple dosage model: Y ~ A + B + AB where Y is the phenotype, in this case, gene expression values and A and B are vectors of genotype ...
469 views

### Paired permutation test for repeated measures

I have a data set consisting of interactions between male-female dyads within a group in two conditions. The values range from -1 to 1 and indicate how responsible the female was for maintaining the ...
517 views

### Permutation tests with repeated measures

Although reading quite a bunch of books, I'm still not sure, which method to use and how to implement it, therefore I appreciate any help! I have 4 different groups (treatments) with 50 participants ...
2k views

### Friedman's test and post-hoc analysis

I am carrying out a statistical analysis for my research. I am using a Friedman's test with post hoc analysis. At present I am using the function ...
498 views

### Random permutation test for feature selection

I am confused about permutation analysis for feature selection in a logistic regression context. Could you provide a clear explanation of the random permutation test and how does it applies to feature ...
1k views

### Correlation analysis and correcting $p$-values for multiple testing

I am investigating the effect of a drug on EEG and cognition in epilepsy patients. We have done EEG and neuropsychological (NP) tests twice, before and after medical treatment. Some of parameters in ...
8k views

### Which permutation test implementation in R to use instead of t-tests (paired and non-paired)?

I have data from an experiment that I analyzed using t-tests. The dependent variable is interval scaled and the data are either unpaired (i.e., 2 groups) or paired (i.e., within-subjects). E.g. ...
349 views

### The probability for two people to provide identical answers on survey questions

Here is the problem: A survey contains 7 binary questions (Yes/No responses). If two people are answering the survey, what is the probability for their answers on 4 or more of the questions to match? ...
761 views

### Help with a one-sample permutation z-test

This is related to another question I asked recently. To recap: [I had 30 people call a number and then roll a 5 sided die. If the call matches the subsequent face then the trial is a hit, else it is ...
416 views

### Should I use an average ECDF?

This relates to a previous question of mine which didn't gain many responses, perhaps because it wasn't very clear nd well written. I hope this time I will be more accurate and get your much ...
400 views

### Random permutation of a vector with a fixed expected sample correlation to the original?

Suppose you have an $n$-vector $X$. For a fixed real number, $r$ between $-1$ and $1$, can one generate a random permutation of the integers $1,2,\ldots,n$, call it $i_1,i_2,\ldots,i_n$ such that the ...
1k views

### Can I use permutation tests for to avoid the multiple comparison problem in the context of proportions?

I am evaluating the effectiveness of 5 different methods to predict a particular binary outcome (call them 'Success' and 'Failure'). The data look like so: ...
660 views

### What is wrong with this “naive” shuffling algorithm?

This is a follow-up to a Stackoverflow question about shuffling an array randomly. There are established algorithms (such as the Knuth-Fisher-Yates Shuffle) that one should use to shuffle an array, ...