Questions tagged [permutation-test]

Statistical tests based on rearrangements of data that are consistent with the null hypothesis.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

What is a permutation test?

I am trying to understand what a permutation test in a perMANOVA is. Can someone either explain how data is being substituted for each permutation or direct me to a good visualization on youtube?
0
votes
0answers
8 views

Which is the best approach for robustness testing for deep learning models?

I tried 10 fold CV and my validation accuracy is above 95%. Just to check the robustness of my CNN model I wanted to do a permutation testing. But is that necessary?
0
votes
1answer
25 views

Which is the best approach for permutation testing for deep learning models?

Should I just shuffle my labels and train my model for a certain number of iterations? The result of permutation testing should give low testing/validation accuracy, right? Is it necessary to do a ...
0
votes
0answers
10 views

adonis vs. adonis2

I am trying to conduct permanova tests on a dataset. There are multiple options for doing this in R and I am not sure how to decide what to use. What is the difference between adonis() and adonis2() ...
0
votes
0answers
36 views

When distributions are non normal and heteroscedastic, is it preferred to use ordinal logistic regression or permutation tests?

I am currently conducting statistical tests on my two independent samples (both with more than 1500 entries each). The sample sizes are no equal. My response variables are interval as well as quasi ...
0
votes
0answers
11 views

Interpreting results of ANOSIM conducted on non-abundance data

I am trying to understand how to implement and interpret an analysis of similarities (using the vegan package in R) in the following situation. Consider the following data frame: ...
0
votes
1answer
21 views

'one-way anova' randomization test in R

I want to implement basic randomization tests as described in Ernst, 2004 Permutation methods for exact inference. I have observations of a single continuous variable for n individuals falling into k ...
0
votes
0answers
23 views

Pearson correlation not distributed around zero when using nonnegative least squares (NNLS) in permutation test?

I am trying to explain artificially created fmri data in MATLAB [1] through a linear combination of w neural network layers. To compare those, I transformed the data into respresentational ...
2
votes
1answer
32 views

Selection of test statistic for permutation test of variability

I have two arrays, $x_1$ and $x_2$ and want to see if there is a statistically significant difference in variance between them. A permutation test makes sense, where the test statistic would be the ...
1
vote
0answers
25 views

Can a permutation test be used on time series to quantify an effect of the marketing campaign?

I am trying to accept or reject a hypothesis that a marketing campaign had positive effect on sales of some product. I am working with time series of monthly sales. The series is not stationary (it ...
6
votes
3answers
239 views

Permutation tests and exchangeability [duplicate]

Permutation tests assume exchangeability of the response/observations under the null hypothesis. In what practical situations is this clearly violated? When is it unproblematic? Edit/additional ...
1
vote
1answer
34 views

Clarification of methods for using non-parametric tests to compare groups- vegan package

I would like to validate my approach to comparing groups using non-parametric tests. data created below describes the structure of my data. Each ...
1
vote
0answers
13 views

Control of type I error rate in functional ANOVA F-test

I am following how to compare curves via functional data analysis, specifically functional ANOVA. This video makes sense in describing it as being a lot like vanilla ANOVA...up to a point. When we do ...
0
votes
0answers
10 views

Dealing with duplicated values in points in pearson correlation testing

I am looking to analyze the correlation between the body weights of people and their body fat percentages. There are two difficulties in my data: the first is that I have multiple data points per ...
3
votes
1answer
27 views

Testing for equality of variance for more than 2 groups

How do you test for equality of variance between, say, 3 groups? Is there an extension of the two-sample F-test? Or is it OK to just run multiple F-tests or permutation tests among 3 groups?
2
votes
1answer
58 views

Mann-Whitney with repeated measures on only *some* participants

I've been working on the stats for a paper which involves comparison of various pairs of groups on a particular score that is heavily skewed (and so I don't feel it would be well suited to comparison ...
0
votes
0answers
22 views

RDA non-significant RDA axes

I have looked at online forums and through various papers and am a little stumped on the interpretation of my results for RDA analysis. I ran the full model and came up with a significant model with ...
1
vote
0answers
15 views

Proper length of random nonce for hash calculation (blockchain)?

I have a string s and I need to calculate a nonce such that when appending the nonce to the string, the generated hash starts with a given sequence. The hash has 256 bits, so the number of ...
1
vote
1answer
58 views

Can we use permutation test everywhere to check difference when the null hypothesis is “there is no difference between two groups of data?”

What I am trying to do? Let's assume, I am working on a research problem where only one participant is present. I have a data set consisting that participant's respiration rate (The number of breaths ...
0
votes
0answers
13 views

How to select a feature selection method?

Is there any other way to evaluate feature selection quality than model performance? I worry that if my model is overfitting this will then bias which selection of features seems to be best. I use ...
3
votes
0answers
69 views

Is permutation test superior to t-test in all cases?

All other things being equal is there ever a time when a t-test would be preferable to a permutation test? Computational time may be one reason, but for our purposes assume it does not matter. Isn't ...
0
votes
1answer
24 views

how should I adjust P values for multiple comparison from permutation test?

I have around 1000 permutation test and get a list of p values. Not sure if it is ok to use P.adjust(p_values) in R to directly adjust my permutation p values? I know in other settings(like t-test) I ...
1
vote
0answers
12 views

Is there the equivalent of the stepAIC function for PERMANOVA?

I am comparing the presence and abundance of species between different sites, which are different in more than 1 factor, and the factors are not independent. I am looking to get the relative ...
0
votes
0answers
38 views

How to test whether the simulated mean differs from observed mean?

I have three log-return time series data1, the data look like: ...
0
votes
0answers
15 views

Not sure which stats to use in SPSS!

I have a set of data collected on soap usage measured in mm depending on 2 options of signs in bathrooms. The data is not normally distributed and I want to test to see if the differences are ...
1
vote
0answers
194 views

The way PERMANOVA handling continuous explanatory variable

I have read some tutorials which explain the mechanism behind PERMANOVA, e.g., enter link description here. Based on the tutorials, my understanding is PERMANOVA can only handle categorical ...
0
votes
0answers
42 views

Is it correct to use permutation testing to find the significance of a clustering solution?

I am performing exploratory clustering with k-means in a multidimensional (82x18) dataset. The algorithm that I am using tests several number of clusters and uses several goodness of fit metrics to ...
1
vote
0answers
45 views

Restricted Permutations for Repeated Measures Nested Plot PERMANOVA

Apologies, for this might be somewhat convoluted. I have an experiment examining community-level differences among treatments and are using PERMANOVA along with other multivariate methods to do this. ...
1
vote
0answers
43 views

What to do if an exact permutation test is infeasible

For permutation tests, it is often infeasible to enumerate all permutations. There are two approaches to solve this. The first uses a random number of permutations instead of all permutation (this is ...
1
vote
0answers
57 views

Permutation Tests: How to Interpret a Non-zero-centered Null Distribution

In a "textbook" experiment (e.g., a single treatment is randomly given to a subset of independent units, the SUTVA holds, etc.), my understanding is that a permutation test of the experiment should ...
0
votes
0answers
55 views

Should I include all the covariates in a PERMANOVA test (R)?

I have a matrix with abundance ( df_for_abundance) from gut microbiota and a matrix with metadata (26 variables; meta). I would like to calculate the effect of clinical covariates on the composition ...
1
vote
0answers
62 views

Wilcoxon signed-rank test on cross-validated values

Many neuroscience papers aim to estimate the significance of classification accuracy measures (AUC-ROC for example): can one classify such behaviors better than chance with such neuronal activities? ...
0
votes
0answers
22 views

Valid significance testing of coefficients when performing model selection

I want to know how I can validly perform permutation (or bootstrap) significance testing of model coefficients when I am also using the same dataset to perform model selection. My model is not the ...
0
votes
0answers
17 views

lunar phase influence on animal abudance in stomach content of predator species

I have a set of data presenting the abundance of prey in the stomach of one predator species. each predator was identified with an ID number and date was recorded among other info. The date was used ...
2
votes
0answers
43 views

Can we perforn a permutation test with zero-inflated data?

In a collaborative problem-solving task, I would like to check if the mean number of communicative exchanges emitted by a person is different following whether they shared an emotion just before or ...
0
votes
0answers
491 views

Interpreting results of resemblance-based permutation methods: PERMANOVA and PERMDISP?

I am working with an invertebrate data set (i.e., counts of individuals per invertebrate order, captured by pitfall trap) and am exploring trends in community composition in relation to environmental ...
1
vote
0answers
24 views

Hypothesis tests via permutation importance on validation data

A model agnostic way to measure variable importance of a variable $x$ in a statistical model (originately proposed by Breiman) is as follows: Select a scoring function $\mu$ to measure model ...
3
votes
1answer
50 views

Examples for exact tests which are not permutation tests

Wikipedia states (without citation): Sometimes permutation test is used as a synonym for exact test, but although all permutation tests are exact tests, not all exact tests are permutation tests. ...
0
votes
1answer
27 views

Number of all sample divisions in permutation test

I have a question with regard to the number of permutations in a permutation test. Assume I want to do a two group independent t-test. In the permutation variant of the t-test, I would repeatedly ...
0
votes
1answer
29 views

Do all non-parametric statistical tests involve bootstrapping, simulation, or permutation test?

All of the non-parametric tests I learned about involved bootstrapping, simulation of large samples of some random variable, or permutation tests. Are there non-parametric tests without simulation or ...
0
votes
0answers
37 views

Bootstrapping of data, in R, for improving the homogeneity of variance

I have a data collected on five point likert scale; aims at studying, the level of preference individuals gives to the different factors, (such as Tax Benefits, Maturity period, Rate of Returns ...) ...
2
votes
0answers
133 views

How permutation based FDR control methods take into account the correlation structure?

There are a number of permutations-based False Discovery Rate (FDR) control methods, such as SAM [1], dChip [2], Ge et al [3], Meinshausen [4], and Efron [5]. It's been said that in these methods, ...
0
votes
0answers
196 views

PERMANOVA test and its assumptions

I have collected data based on a 5 point Likert scale (very low, low, neutral, high, very high) on factors considered by individuals before making investment decisions. There are five factors (D.Vs) ...
0
votes
0answers
65 views

Permutation Manova

I have collected the data based on 5 point likert scale(very low, low, neutral, high,very high) on the factor considered by individual before making investment decision. There are five factors (D.Vs) ...
4
votes
3answers
142 views

Is there a formula to calculate all possible unique permutations of n elements over p positions?

Is there a formula to calculate all possible unique permutations of n elements over p positions? Please imagine the following ...
0
votes
1answer
40 views

Sampling from the null distribution for a t-test

I am trying to apply the Dietterich 5x2 cv t-test to test whether one statistical model is better than another on a particular data set. In order to apply this test, one must assume that under the ...
0
votes
0answers
57 views

Did I appropriately run a permanova on an additive model with nested factors in R?

Because I don't have a statistician readily at my disposal and I am still fairly new to multivariate statistics, I hope maybe one of you out there could maybe check whether I did an appropriate ...
0
votes
1answer
32 views

How can I generate a completely balanced combination

Attaching a code that generates unique combinations of 8 letters but, it still lacks a condition wherein all letters should have equal counts per column. For my code, basically, letter a to h should ...
0
votes
1answer
210 views

In VEGAN ADONIS, what does "method='bray' do when I pass in a distance matrix I already have (UNIFRAC)?

Suppose I have a distance matrix created by applying the weighted unifrac measure. For instance, using a phyloseq object: ...
0
votes
0answers
4 views

Example of Permutations of Objects which are Alike [duplicate]

Can you give an example that let me visualize the formula of the number of permutation of n objects in which k1 are alike, k2 are alike, etc. n!/k1!×k2!×...×kn! where k1! + k2! + ... + kn = n Also ...

1
2 3 4 5
8