# Tagged Questions

A permutation is a rearrangement of values, items or objects, which arise in various statistical contexts. Permutation tests are statistical tests based on all possible rearrangements of data that are consistent with the null hypothesis.

31 views

### How to calculate permutations of categorical variables with R

I want to simulate or calculate probabilities of combinations of group membership for different sample sizes (e.g., n= 3, 4, 5, 10, or 100) for two groups (of the same sample size). Each outcome could ...
23 views

### Hypothesis testing for equivalence of two arrangements

I have two arrangements(i.e. permutations) of numbers. First one is the target/real arrangement. Second, is the observed arrangement. e.g. Target := 1,2,3,4,5,6,7 Observed := ...
30 views

### Repeated Measures GLM / restricted permutations

I have a question regarding repeated measures and GLMs: Suppose i have counted the abundance of some species in lakes at different time points - every lake received a different treatment at time = 0. ...
10 views

### GeneralLM, how different are these three design matrix?

I have some data having structure as like below ...
65 views
+50

### Correlation between two Decks of cards?

I've written a program to simulate an overhand card shuffle. Each card is numbered, with suit going from CLUBS, DIAMONDS, HEARTS, SPADES and the rank from Two ...
17 views

### Strange intercept using lmp() in lmPerm library in R

Please consider this data set: y <- c(2, 4, 6) x <- c(1, 2, 3) Now calculate a linear model using lmp(): ...
7 views

### enrichment test and relations between variables

I'm in a situation in which I derived a list of 500 genes using a sophisticated method, of which I found a subgroup of 60 genes that are mutated according to a mutational annotation. To verify the ...
132 views

### Question re permutation tests for a relatively complex data set

I regret that I cannot be overly explicit due to confidentiality agreements. I will try to state my problem by analogy. Suppose that you have two groups of people, one with a condition and one ...
126 views

### What is the intuition behind exchangeable samples under the null hypothesis?

Permutation tests (also called a randomization test, re-randomization test, or an exact test) are very useful and come in handy when the assumption of normal distribution required by for instance, ...
94 views

### How to figure out what numbers often appear together in a dataset?

I have a lottery style dataset we produce internally (example below). I am trying to figure out which numbers appear most frequently together. Example questions: What are the top 10 pair of numbers ...
58 views

### How to report a permutation test?

I want to compare a parameter between two independent groups of volunteers (see MWE below). All volunteers had baseline reading (V1) and a subsequent reading by a later visit (V2) for intervention ...
175 views

### Probability of a given password

A password consists of 4 alphabet letters and 4 numbers. Calculate the following two probabilities: $p_1$: the probability that the letters are all equal and that the numerical part contains one ...
52 views

### Permutation test of meta-analysis : correlation coefficient

I am going to estimate False Discovery Rate using permutation tests. To my knowledge, several R packages are applicable for multiple testing. I have several independent datasets for meta-analysis. ...
38 views

### Probability with fair dice [duplicate]

If you roll 5 standard 6 sided dies, what's the probability that you will get at least three 2s? I imagine it would be 1 - P(0 twos) - P(1 two) - P(2 twos), but I don't know how to calculate the ...
40 views

### Permutation test: Exactness

I am currently reading about permutation/randomization tests and have some difficulties to understand why they are exact. More precisely, I consider two groups of random variables with means $\mu_1$ ...
41 views

### Two sided permutation test

I'm trying to write a two sided permutation test to test the alternative hypothesis there is a difference in the medians of 2 independent samples. My question is this: am I calculating the p-value ...
35 views

74 views

### Permutation test where Xi stays with Yi, but the calculation that results in Xi is permuted

I've tried to find a good answer for this by searching this and other sites, but couldn't, so forgive me if something like it has been answered. I am doing a linear regression with an independent ...
66 views

### Permutation test comparing a single sample against a mean

When people implement permutation tests to compare a single sample against a mean (e.g., as you might do with a permutation t-test), how is the mean handled? I have seen implementations that take a ...
132 views

### Non-parametric tests comparing dependent samples

Suppose I have two samples S1 and S2, but they are dependent samples. My aim is to compare if the mean of S1 is significantly smaller than the mean of S2. The samples are not normally distributed, so ...
95 views

### Alternative to the t-test with dependent samples

This might be a simple question, but just wanna make sure I am on the right track: I need to compare two samples using a 1-sided t-test. The sample sizes in the two groups are unequal, but it should ...
115 views

### Randomisation/permutation test for paired vectors in R

I'm not an expert, so forgive me if some of the terminology is a little clumsy. Happy to provide more information where required. I have two vectors of 50 paired numeric values in R. I want to ...
38 views

### Good reference for number of permutations for a simple test?

I have two vectors of 100 real values each. I would like to use a permutation test for a few things such as the correlation of the two vectors. I would use the approach of keeping one vector ...
45 views

### What role can permutation test play in multiple comparison?

In a test, permutation test is used to estimate the null distribution of the test statistic from the sample. In multiple comparison, what is the usage of permutation test? Thanks!
109 views

### Is the p value in a permutation test the same as the p value in the original test?

This is an extension from my last post. It seems to be too long to discuss there. When testing some null $H$ versus alternative $K$ by a test statistic $U(X)$, the p-value for $U$ on a sample $X$ ...
219 views

### What is the benefit of using permutation tests?

When testing some null versus alternative hypotheses by a test statistic $U(X)$, where $X = \{ x_i, ..., x_n\}$, apply the permutation test with the set $G$ of permutations on $X$ and we have a new ...
162 views

### Implementing a Kolmogorov-Smirnov permutation test

I would like some advice on how to implement a Kolmogorov-Smirnov test. This is a condensed version of my previous question -- any further detail can be provided if needed. I have data from n ...
77 views

### Testing for non-random overlap of polygons

I have two polygon shapefiles and I want to see to what extent the observed amount of overlap is due to chance. I'm thinking of some kind of permutation test, but not sure of the best way to ...
70 views

### Permutation test null and correct label distribution

Permutation tests can be used to test the likelihood of obtaining a single mean value (composed of the average of k-fold CV) classifier accuracy by chance, such as for a linear classifier using two ...
66 views

### Calculating card deck shuffling odds

Start with a standard deck of 52 cards in some known order; each card can be tagged with its ordinal position in the deck. Then apply a random shuffling algorithm (i.e. Knuth's shuffle). After ...
55 views

### A stopping rule for permutation test

I perform a permutation test multiple times on different datasets, each time I am only concerned about significant $p$ values. To reduce computation time would it be correct to introduce this kind of ...
75 views

### Non-parametric test for comparing four treatments and three times

I am working with a dataset where I need to use non-parametric stats (won't go into details). It is behavioral data on captive animals ($n=8$), where $4$ treatments were introduced $3$ different times ...
226 views

### Permutation test for multiple correlation test statistics

I read some great discussions about using permutation tests on correlation matrices to deal with Type I errors that arise from the multiple comparisons; however I have a question about the correlation ...
57 views

### Does permutaton test change the null hypothesis of the original test?

Suppose we have a test statistic designed for testing a null about two groups of samples. If we apply permutation test to the groups of samples and the test statistic, will that change the null ...
63 views

### How does adding a “lives” mechanic change average picks?

I'm tuning a slot machine app and need to make sure the mini-game is appropriately balanced. The mini-game is a grid of 30 treasure chests, 6 of which are traps. The user continues to open chests ...
131 views

I've been spending quite some time to figure out how I can get the best R squared value from randomization of some values in a linear regression equation. I have allele frequency data and 14 ...
524 views

### Difference between Randomization test and Permutation test

In the literature the terms Randomization and Permutation are used interchangeably. With many authors stating "Permutation (aka randomization) tests", or vice versa. At best I believe the difference ...
70 views

### Why do lmp (from lmPerm package) and boot results vary widely

Using built in states data set I fit lm1<-lm(Murder~Illiteracy*Income), lm2<-lmp(Murder~Illiteracy*Income), results<-boot(...,formula=Murder~Illiteracy*Income) lm1 and result coefficients ...
113 views

### Tests for quality of permutation (Y-randomization)

Can you please suggest the metrics/tests for accessing the quality of Y-randomization (permutation)? I have some doubts in quality of Y-randomization that I am using with sample.int from R. For 100 ...
390 views

### How to compute p-value from a permutation test?

I'm running a series of permutation tests inside 2 for loops (2 for loops will calculate some new data and then this data will be shuffled and then i apply a linear regression based on the shuffled ...
202 views

### Resampling with replacement permutation analysis

My question might be easy for most of you to answer. I am starting to learn statistics and coding in R so my questions are on the basic level. I have a dataset of (two groups, replicates). My data is ...