The permutation tag has no wiki summary.
0
votes
1answer
33 views
Permutation test null and correct label distribution
Permutation tests can be used to test the likelihood of obtaining a single mean value (composed of the average of k-fold CV) classifier accuracy by chance, such as for a linear classifier using two ...
2
votes
1answer
45 views
Calculating card deck shuffling odds
Start with a standard deck of 52 cards in some known order; each card can be tagged with its ordinal position in the deck.
Then apply a random shuffling algorithm (i.e. Knuth's shuffle).
After ...
3
votes
1answer
35 views
A stopping rule for permutation test
I perform a permutation test multiple times on different datasets, each time I am only concerned about significant $p$ values. To reduce computation time would it be correct to introduce this kind of ...
1
vote
0answers
41 views
Non-parametric test for comparing four treatments and three times
I am working with a dataset where I need to use non-parametric stats (won't go into details). It is behavioral data on captive animals ($n=8$), where $4$ treatments were introduced $3$ different times ...
0
votes
1answer
69 views
Permutation test for multiple correlation test statistics
I read some great discussions about using permutation tests on correlation matrices to deal with Type I errors that arise from the multiple comparisons; however I have a question about the correlation ...
2
votes
2answers
48 views
Does permutaton test change the null hypothesis of the original test?
Suppose we have a test statistic designed for testing a null about two groups of samples.
If we apply permutation test to the groups of samples and the test statistic, will that change the null ...
1
vote
1answer
58 views
How does adding a “lives” mechanic change average picks?
I'm tuning a slot machine app and need to make sure the mini-game is appropriately balanced.
The mini-game is a grid of 30 treasure chests, 6 of which are traps. The user continues to open chests ...
0
votes
1answer
76 views
Randomizing variables to get the best combination for high R-squared values in R
I've been spending quite some time to figure out how I can get the best R squared value from randomization of some values in a linear regression equation. I have allele frequency data and 14 ...
3
votes
3answers
176 views
Difference between Randomization test and Permutation test
In the literature the terms Randomization and Permutation are used interchangeably. With many authors stating "Permutation (aka randomization) tests", or vice versa.
At best I believe the difference ...
0
votes
0answers
47 views
Why do lmp (from lmPerm package) and boot results vary widely
Using built in states data set I fit
lm1<-lm(Murder~Illiteracy*Income),
lm2<-lmp(Murder~Illiteracy*Income),
results<-boot(...,formula=Murder~Illiteracy*Income)
lm1 and result coefficients ...
0
votes
1answer
68 views
Tests for quality of permutation (Y-randomization)
Can you please suggest the metrics/tests for accessing the quality of Y-randomization (permutation)?
I have some doubts in quality of Y-randomization that I am using with sample.int from R. For 100 ...
0
votes
1answer
262 views
How to compute p-value from a permutation test?
I'm running a series of permutation tests inside 2 for loops (2 for loops will calculate some new data and then this data will be shuffled and then i apply a linear regression based on the shuffled ...
3
votes
1answer
130 views
Resampling with replacement permutation analysis
My question might be easy for most of you to answer. I am starting to learn statistics and coding in R so my questions are on the basic level.
I have a dataset of (two groups, replicates). My data is ...
0
votes
0answers
199 views
how to calculate gini variable importance in random forest?
I am curious about how gini variable importance is calculated in R. Is it something like:
for one node, calculate gini index $ 1-\text{branch}_1\%^2-\text{branch}_2\%^2; $ then calculate gini index ...
6
votes
2answers
575 views
Permutation test in R
I have the following data for 10 subjects based on before and after measurements:
...
1
vote
0answers
246 views
How to report permutation ANOVA results in a scientific publication?
I have been using lmp function from lmPerm package to calculate differences between treatment levels (2-ANOVA). Now, I am wondering how to report ...
2
votes
0answers
72 views
is it valid to take a mean of p-values during cross-validation, when comparing the predicted output of a model to the actual output?
I am doing a cross-validation study, training a model on an input to predict a target.
During training, my model generates an output vector that is guaranteed to be the same size as the corresponding ...
1
vote
1answer
124 views
Permutation test for the significance of several dependent regression coefficients from separate models
I have $i>100$ Cox regression models which differ only in one explanatory variable, $x_i$. They have the same outcome data and the same other explanatory (adjustment) variables.
I want to control ...
2
votes
1answer
97 views
Controlling type I error in permutation tests
I'm implementing a permutation test for a test statistic with unknown distribution. Apart from my concrete situation (see below), I'd like to know first
If and under what circumstances permutation ...
2
votes
1answer
109 views
Permutation paired test with signed statistic
I have one paired data from 7 patients:
$$(X_i ,Y_i)=\{(5,1),(-2,-4),(4,2),(-6,-3),(-3,-3),(-5,2),(3,1)\},\quad (i=1,\ldots, 7)$$
where $X$ means placebo $Y$ means the test drug.
My classmate asked ...
2
votes
1answer
133 views
How many random permutations to cover all possible permutations?
I have code that generates a random permutation. In my case, a permutation consists of N binary features, and each of the N features is set or unset randomly. How many times must I generate a random ...
2
votes
1answer
139 views
Permutation test to test significance of skewness/kurtosis of two distributions?
To test the significance of the skewness difference between two distributions with $N_1$ and $N_2$ samples, would the following test work:
Create a single array of all the samples from both ...
2
votes
0answers
172 views
lmPerm p-values and multiple testing
I've started using lmPerm in order to perform regressions in R.
The equation I want to fit has the form:
...
12
votes
2answers
263 views
What is this permutation?
I was recently looking for ways to resample time series, in ways that
Approximately preserve the auto-correlation of long memory processes.
Preserve the domain of the observations (for instance a ...
0
votes
0answers
39 views
How can I determine if one animal is following another?
I have 90 videos of animals arriving at a location and want to determine if one species is following another.
I guess I need to look at the probability that the two are within are certain time of ...
1
vote
1answer
82 views
Citation for permutation test
I am using a permutation test to verify the statistical significance of an improvement in multi-class classification performance. What citation would be appropriate?
Would it be fine to cite ...
1
vote
0answers
109 views
Limitation on group size with non-parametric permutation tests
I would like to know if there is any limitation in the size of the groups when you are implementing non-parametric permutation tests. My case belongs to the realm of medical imaging. I am comparing ...
3
votes
1answer
393 views
Two-sided permutation test vs. two one-sided
I am working with a set of genes for which I have both methylation values $\beta$ (continuous on unit interval) and gene expression $E$ (non-negative continuous) that I want to test for correlation ...
3
votes
1answer
139 views
Correlation analysis and correcting p-values for multiple testing II
I am currently facing problems dealing with lots of individual simple correlations (28 individual DTI connections strengths, individual behavioural scores on 31 time points). That would be 28x31 ...
0
votes
0answers
61 views
Unique subsequences of the human genome
I am working on a project involving genomes. I am going to divide a genome into subsequences, and then do something with those subsequences. I want to know if the subsequences that I produce are going ...
0
votes
0answers
76 views
Analyzing ratios or numerator/denominator, with weights, measured over time at irregular intervals
I have gathered information on the rate increases allowed by a government regulator over about 17 years. As background, utilities in this area must apply to the regulator if they wish to raise their ...
1
vote
1answer
104 views
Monte Carlo test for comparing curvature of binomial response surfaces from effective degrees of freedom of GCV-fitted splines
I would like to compare the curvature of two response surfaces, each of the form:
binomial ~ continuous variables1-5
I think it would be appropriate to use the effective degrees of freedom of a ...
1
vote
1answer
574 views
Permutation tests in R for correlations
What's the easiest way to run permutation tests for a bivariate Pearson correlation matrix in R?
I have tried running MPT.Corr, Multivariate Permutation Test
for Correlations by Urbano Blackford et ...
3
votes
1answer
175 views
Doubling the tails in two-sample permutation test
Suppose we have two samples and we wish to determine if they are drawn from the same distribution, the samples A,B being composed of some integers say.
If we test this using a two-sample permutation ...
1
vote
2answers
105 views
To what degree is FWE correction required for multiple permutation based t-tests
I am performing multiple t-tests to compare conditions of data. I have had to resort to permutation based t-tests (based on difference of means) due to high variance between samples.
My question is ...
7
votes
3answers
997 views
How to resample in R without repeating permutations?
In R, if I set.seed(), and then use the sample function to randomize a list, can I guarantee I won't generate the same permutation?
ie...
...
1
vote
1answer
616 views
T-test vs. Permutation test on (not surely) normal data
This question follows my previous one.
I will now provide my data in order to help you help me :)
As previously said, I have N=42 from a controlled experiment.
People made a questionnaire assessing ...
4
votes
1answer
276 views
Restricted permutations
I have some multivariate data and want to investigate the effect of some environmental gradient. I want to use capscale but I don´t know how to deal with the permutation scheme. I have made up some ...
0
votes
1answer
217 views
How to define the rejection region for a permutation test?
I am trying to find out the rejection region for a permutation test about the difference between medians where:
...
5
votes
1answer
683 views
Is there a way to compare linear regression slopes by permutation tests?
I have some species $\times$ year matrices and analyse the temporal dynamics by regressing distance between years on $\sqrt{ {\rm time \ lag} }$. Since the data points are not independent and since I ...
3
votes
1answer
470 views
Generating all possible unique combinations of dataset with two variables
I am fairly new to R and I have been reading all related topics but still I don't manage to solve my rather simple problem.
I have a dataset with 5 cases for two variables (X,Y), for instance:
...
1
vote
1answer
227 views
NAs in summary of linear model generated from significant genes for continuous response variable identified using SAM
I have a gene expression data-set with log2-transformed expression values (no NAs) for 495 genes for 59 samples for which values of a continuous response variable (r) are also known (no NAs). I want ...
4
votes
2answers
1k views
How to generate nD point process?
Context:
Poisson point processes (PPP) are widely discussed in the literature.
In the following figure a framework to generate two-dimensional PPP is demonstrated. First the area being studied (part ...
2
votes
1answer
597 views
Bootstrap and randomization tests to compare paired data sets
I have 50 values set as "true" in a simulation and attempts to recover these "true" parameters using 2 different models. Here's 3 (instead of 50) runs just to show the data layout (these are made up ...
3
votes
3answers
737 views
False discovery rate from permutation testing?
I have conducted a search for genetic interactions using a simple dosage model:
Y ~ A + B + AB
where Y is the phenotype, in this case, gene expression values and A and B are vectors of genotype ...
2
votes
3answers
430 views
Paired permutation test for repeated measures
I have a data set consisting of interactions between male-female dyads within a group in two conditions. The values range from -1 to 1 and indicate how responsible the female was for maintaining the ...
1
vote
2answers
433 views
Permutation tests with repeated measures
Although reading quite a bunch of books, I'm still not sure, which method to use and how to implement it, therefore I appreciate any help!
I have 4 different groups (treatments) with 50 participants ...
4
votes
1answer
2k views
Friedman's test and post-hoc analysis
I am carrying out a statistical analysis for my research. I am using a Friedman's test with post hoc analysis. At present I am using the function ...
9
votes
1answer
430 views
Random permutation test for feature selection
I am confused about permutation analysis for feature selection in a logistic regression context.
Could you provide a clear explanation of the random permutation test and how does it applies to feature ...
5
votes
2answers
1k views
Correlation analysis and correcting $p$-values for multiple testing
I am investigating the effect of a drug on EEG and cognition in epilepsy patients.
We have done EEG and neuropsychological (NP) tests twice, before and after medical treatment.
Some of parameters in ...
