Questions tagged [permutation-test]

Statistical tests based on rearrangements of data that are consistent with the null hypothesis.

Filter by
Sorted by
Tagged with
0
votes
0answers
28 views

How to test the difference between dependent partial correlations in the presence of autocorrelation?

Background and what I am trying to do: I have a single dependent variable $Y$, and $K$ independent variables of interest ($X_1, X_2... X_k$). I also have a single independent variable of non-interest, ...
0
votes
0answers
18 views

To do a test via bootstrap, should I bootstrap the actual data the shifted (under the null) data or the test statistics?

Please help me to resolve the source my confusion. Let us assume I want to do a test via bootstrap. For simplicity, let it be comparison of two means. I saw the following approaches: to resample &...
2
votes
0answers
23 views

Monte carlo simulation [closed]

I am working with some data with sample size equal to 11, looking for differences between two locations; a and c. Due to the small sample size and the absence of normal distribution I have opted for ...
1
vote
1answer
28 views

permutation test on correlation coefficient

I have a correlation coefficient of two hierarchical clustering trees with 20 labels. I want to check the hypothesis that the labels of the trees are uncorrelated with a permutation test. I calculated ...
0
votes
0answers
11 views

Am I inputting the correct data for randomisation tests for my single-case design in R?

I'm currently doing randomisation tests (RTs) for a single-case design in R. A bit of context about my phases: I am doing an AB design, which was staggered using a multiple baseline design across 5 ...
2
votes
1answer
47 views

Permutation statistics yield different results when computed again

I am doing a within-subject design experiment and trying to analyze my data. To simplify my problem, let's say that there is condition A and condition B, and I have 24 pariticipants. I want to do ...
0
votes
1answer
31 views

Intuition behind permutations + probability

I'm new to stats so I'm struggling to grasp the intuition behind permutations + probability. Would anybody be able to help me with parts (b) and (c) of this question? Any help is greatly appreciated :)...
0
votes
0answers
10 views

compute likelihood statistic with permutation test

This question is about hypothesis testing, where we want to use the likelihood ratio statistic with permutations test. Suppose we sample $n$ observations from the distribution $F_{XY}$, which is the ...
0
votes
1answer
44 views

Doubts with P-value and Hypothesis in Permutation Test problem

I have to solve this problem for a stats course: An e-commerce company is testing a new design for a web page. The objective is to achieve at least a 2% increase in the conversion rate. An experiment ...
0
votes
0answers
26 views

Implementing permutation tests in randomized controlled trials with repeated measures

I am trying to implement a permutation test to use for testing treatment effects in a repeated measures RCT-design. Unfortunately I am having some trouble understanding how to design a correct ...
0
votes
0answers
30 views

Alternatives of permutation test to proof that a model performs better than chance level

I built a model based on a very high dimensional data, which requires 4-5 days to built using High Performance Computing (HPC). I want to do a statistical test to proof that the predictive power of my ...
0
votes
0answers
4 views

Difference between PERMANOVA and Multiple Regression on distance Matrices

I am a newbie in statistics so the math part of these 2 tests might not be crystal clear. But it seems that they are quite similar if you try to analyze e.g. how 4 variables explain the dissimilarity ...
0
votes
1answer
75 views

Permutation test and wilcoxon rank-sum test giving different results

I am comparing the revenue (at user level) between control & test to see if there is a significant difference. Both control & test have roughly about 39k data points . I am doing a permutation ...
0
votes
0answers
14 views

Statistical test for assessing trends (one-sample)

I'm interested in looking at annual trends. The numbers represent the total number of a particular animal species that have been counted in that year. We see a decreasing trend, and I would like to ...
0
votes
0answers
18 views

In multivariate analysis using PERMANOVA and NMDS, is it appropriate to include both predictors in my model?

I am an undergrad student new to much of these statistical tests. I am using NMDS ordination and Permanova testing to analyze variation in cover type (using count data) at different distance classes (...
0
votes
0answers
6 views

Compare the performance of a machine learning informed classification regime against a random one

I am building an active machine learning toolkit to speed up articles' classification for systematic reviews. This method has three steps: Order records heuristically according to some relevant term ...
0
votes
1answer
39 views

Analyzing replicates of contingency tables

Problem statement We would like to measure the effect of treatment on cell differentiation, in other words whether or not specific treatments cause cells to change their cell types. So we treat cells, ...
1
vote
0answers
17 views

Random forest on compound analysis and input data permutation

I am using a random forest model to associate climate variables with a specific type of impact, which is measured as the likelihood of failure (therefore, classification). The choice of random forest ...
3
votes
1answer
83 views

Adjusting P values for multiple comparisons using permutation tests

I have a number of continuous predictors (biomarker measurements) which I would like to test for association with a binary outcome variable (disease status), adjusting for multiple comparisons. As ...
0
votes
0answers
31 views

What similarity index should I use with relative abundance data for a PERMANOVA?

I have data on the relative abundance (0 to 1 ratio) of 5 land covers for different landscapes (analogous to species composition). These landscapes are grouped into three geographic areas and I want ...
0
votes
0answers
31 views

PLS model performs significantly worse than chance according to permutation test

I'm using Partial Least Squares regression to relate two datasets: X has 40 variables, Y has 54 variables, both have 26 observations. Because I don't have a lot of observations, I'm using a ...
0
votes
0answers
26 views

Permutation test for non-normal distributions?

I am analyzing if the habitat type (treatment) may affect to the virus incidence. For that aim I performed a permutation analysis to this dataset: ...
0
votes
0answers
48 views

Can the Permutation Test be used for this hypothesis testing problem?

To investigate the association between two categorical variables, some statistical tests exist, for example, the chi-squared test. We didn't cover these test in the course. For example, suppose we ...
0
votes
0answers
23 views

R: vegan/permute: additional level of nesting for pseudoreplicates in a split plot

I am trying to investigate how to set up a permutation scheme for community data originating from a split plot experiment with pseudoreplicates at the lowest level. A theoretical design could be the ...
0
votes
0answers
161 views

how to interpret negative values in permutation importance report?

i am using a permutation report from sklearn. i train my model and optimize on 'mean_squared_error' this is actually negated in sklearn. when i get the perumutation importance report however how do i ...
3
votes
0answers
45 views

What model assumptions are necessary for a changepoint detection based on permutation test

I am writing about the change point detection algorithm suggested by Taylor W. in my bachelor thesis. The full description can be found here: https://variation.com/wp-content/uploads/change-point-...
3
votes
1answer
51 views

permutation test sampling

In statistical inference, specifically while conducting hypothesis testing, we use permutation test, is there any reason why resampling is done without replacement (shuffle), what difference does it ...
1
vote
0answers
34 views

Combining tests: Maximum test statistic method but one test is a permutation test

I want to combine two tests $A,B$ for the same null and alternative hypotheses. The motivation is that it has been shown that test $A$ has higher power in some situations whereas test $B$ has higher ...
0
votes
0answers
24 views

R: When to set bias.adjust = TRUE in vegan::betadisper?

I am trying to test if my data have homogeneous variance using vegan::betadisper() in R. I read the help for the function but it only says that is appropriate when ...
0
votes
1answer
41 views

Combining p-values from permutation tests from single subjects

I have a question that's very similar to this question: Combine multiple independent permutation tests My 'issue' is exactly the same: I have 20 subjects and I used permutation tests to determine the ...
0
votes
0answers
46 views

Permutation test - Median vs Mean

I am trying to study the influence of a Treatment Model (denoted by 1) vs a control group (denote by 0). The goal of my statistical analysis is to be able to say with a certain confidence that the ...
0
votes
0answers
26 views

Correct way to implement the H0 of no correlation between random variables X and Y with a permutation test?

I want to test whether a measurement error corrected correlation between variables X and Y is significantly different from 0 or not. I have 65 paired observations (x1,y1), (x2, y2),...(x65, y65). Due ...
0
votes
1answer
66 views

Bootstrap/Permutation test for equality of two distributions

In the book "An Introduction to the Bootstrap" the authors state that both the permutation test and the bootstrap test can be used to test $H_0:F=G$, where $F$ and $G$ are two different ...
2
votes
1answer
23 views

Does Variance Reduction really help the Nonparametric Test?

In Online Experiment, Working on Variance Reductions could help a lot for the traditional parametric tests like test two-proportion (CTR) or two-mean t-test, as it significantly improves the power and ...
0
votes
0answers
46 views

Does permutation test require iid?

Suppose I have a treatment group X and independent from control group Y. The data set x1,x2..., y1,y2... Let's say I want to test the mean of the difference If the randomization unit is a session ...
0
votes
0answers
17 views

Permutation testing and covariate/confound inclusion random statistics permutation

I have a data set with two groups where participants in both groups were collected from numerous sites. I performed randomised permutation testing between the groups. However, I was asked if I ...
0
votes
0answers
36 views

A (new?) bootstrap-based technique to check if a model performs worse on “subpopulation”

This is sort of an ad hoc statistical test I've devised to assess the performance of a prediction model on different subpopulations I have a large sample A with a Subgroup R. I would like to tell if ...
2
votes
1answer
668 views

Alternative to Bonferroni Correction when performing multiple one-vs-rest association tests

My question anonymizes the 'nouns' of the question to protect my employer. It is not really about lab rats and experimental treatments. I am also coming more from a Machine Learning background, so my ...
1
vote
0answers
59 views

Permutation test - can I apply it here?

I am struggling to understand if I can use a permutation test on my data. I have the following setup: A spine with three vertebral bodies (Level). Fat rings were added to the spine to simulate 5 ...
0
votes
0answers
120 views

Is it possible to use a PERMANOVA (adonis / R) on a time series - are there any alternatives?

I'm having trouble with fitting a PERMANOVA (adonis) to my data and am actually not sure if it is possible at all. I have a pitfall arthropod data set for 5 research areas and 3 consecutive field ...
0
votes
0answers
20 views

Permuting outcome variable to estimate null likelihood ratio distribution for Poisson GLM

I fit a Poisson GLM using a set of candidate explanatory variables ($\lambda = e^{\beta_{0} + \beta_{1}x_{1}+\beta_{2}x_{2}+...}$), tested against the global null hypothesis ($\lambda = e^{\beta_{0}}$)...
2
votes
1answer
67 views

Appropriate practice for setting seeds in permutation analyses?

I am currently using permutation ANCOVA for a study of mine, in R, using the permuco package. Running multiple models for different brain regions of interest (ROIs). What I am finding is that the p-...
0
votes
0answers
87 views

Permutation test for A/B testing

I have done an extensive survey of testing methodologies in A/B practices. It seemed that default non parametric test choice is MWW U test. However, permutation test is known to generally have a ...
0
votes
0answers
50 views

When conducting multiple comparisons with adonis, the p-values of the comparisons should be used as they are or used it after correction?

I recently ran 60 tests with adonis and found some significant differences (p-vaue < 0.05) between control and treatment (I only have one control and one treatment and do not need to do the ...
2
votes
1answer
57 views

Permutation Hypothesis Testing procedure

I'm not strong in statistics and I'm looking for a help. I work with real estate data and I want to compare apartment prices in 2 districts: district "A" and district "B". Data ...
0
votes
0answers
28 views

General procedures to check conditions for hypothesis testing?

I am going to have a linear by linear test(lbl_test in R, and the related doc can be found here) but the math underlying it is still a mystery to me after trying my best to comprehend this ...
0
votes
0answers
13 views

permutation based clustering for fNIRS data in R

I hope this finds the group safe and well. I am working on a problem involving clustering a series of channels. I have a fNIRS dataset, which is the average hemodynamic response across 49 channels ...
0
votes
1answer
33 views

Implement cluster-based permutation test for regression

I have worked with permutation testing previously to assess significance in classification problems. What you commonly do here is to randomly swap group labels to generate a null distribution. However,...
2
votes
1answer
40 views

Getting feature weights with permutation_test_score()

I am using sklearn to fit a SVM to some data. Since I wanted to use cross-validation and evaluate my classification accuracy using permutations, I am using the permutation_test_score() function (https:...
3
votes
0answers
160 views

Which test should I use to compare the effect of a parasite infection on gut microbiome across time serie data point

I am interested in the effect of an intestinal parasite infection on the rat gut bacterial community. I have two groups of rats ("infected" and "non-infected") each consists of 6 ...

1
2 3 4 5
9