# Questions tagged [permutation-test]

Statistical tests based on rearrangements of data that are consistent with the null hypothesis.

432 questions
Filter by
Sorted by
Tagged with
25 views

### In permutation test, why do we take the proportion of sampled permutations with value equal or larger than the observed value?

The tutorial I followed explains permutation testing in an intuitive way. However, it has confused me in one specific part. Why do we take as p-value the proportion/probability of permutation with ...
8 views

### how do I perform permutation testinging for a prediction model developed within caret package (R)?

I'm fairly new to data science/StackExchange, so please excuse any faux pas I'm trying to perform permutation testing for a chosen ML algorithm (an elastic-net logistic regression) to derive a p-value....
38 views

### What is a "permutation" in PERMANOVA?

I'm using PERMANOVA to calculate significant differences between groups in my beta-diversity analysis for 16S rRNA gene sequencing. However, while I understand the results, I don't understand what the ...
6 views

### How to correctly apply a PERMANOVA to a half-crossed experimental design

I'm a biologist and conducted an experiment in which I took analytical samples of aphid infested plants and analyzed them using metabolomics. The plants were chosen based on their Genotype (...
22 views

### Combining two levels of confidence interval/uncertainty

I am doing a random permutation test to estimate the confidence interval for a given statistic based on usual estimates of the Bernoulli distribution. I.e. assuming a normal approximation to the ...
48 views

### Non-inferiority test for Spearman correlation

Let's say I have the following data and want to test whether the Spearman's correlation is > 0.5 ...
45 views

### Distance between two categorical distributions

I want to test whether two (empirical) categorical distributions taking on $K$ possible values (e.g. 5, with no innate underlying ordering) with associated (empirical) probabilities $p_k$ are the same....
51 views

### Permutation test vs Mann–Whitney U test

I am wondering if there exist any advantage of using the Mann-Whitney U test over a permutation test in the following setting: Suppose I have a group of animals with eggs in their brood chamber, from ...
36 views

### Permutation test on dependent data

Say that I have a sample of observations which are pairs of real numbers $(x_i, y_i)$. Pairs are i.i.d., but $x_i$ and $y_i$ are not independent. I want to test whether the mean of $x$ is equal to the ...
23 views

### When does the 'equal' affect most in the permutation test?

First of all, I need to use a permutation test to calculate the significance of some data. When I do the permutation test, I need to shuffle the data and compare the original one and the shuffled one. ...
23 views

26 views

### Does Variance Reduction really help the Nonparametric Test?

In Online Experiment, Working on Variance Reductions could help a lot for the traditional parametric tests like test two-proportion (CTR) or two-mean t-test, as it significantly improves the power and ...