Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.

learn more… | top users | synonyms (2)

0
votes
0answers
6 views

Correction for multiple comparisons repeated measures MANOVA [on hold]

I am running a repeated measures MANOVA and wanted to know the best way to correct for multiple comparisons. If I use Bonferroni, for example, how would I go about doing that within SPSS?
1
vote
1answer
53 views

No significant tests when using Benjamini-Yekutieli multiple testing correction on millions of tests

I am using a univariate filter to reduce the number of features prior to applying a learning algorithm to a huge binary classification dataset (22510066 features x 500 examples). All the features are ...
2
votes
1answer
23 views

What is the best method to analyze this standard example data

The MASS package in R has 'anorexia' data set that compares the results of 3 treatments on weights of patients. Which statistical method is ideal to analyze this data set? There is one control (Cont) ...
0
votes
0answers
15 views

EMMEANS Pairwise Comparison OR Post Hoc Multiple Comparison?

I was running a 2-way Univariate ANOVA, and there are two tables in the results that compair the 3 levels of one of my IVs (Transformation_Model). Under Estimated Marginal Means, the table is titled ...
0
votes
0answers
12 views

How to find significance when performing many significance tests

I have 4 Likert scales in both a pre- and post-survey. Each scale shows significant change from pre to post. I'd like to identify which individual items have a significant change as well. There are ...
1
vote
0answers
17 views

Appropriate test for similarity of four vectors?

I have four samples under four treatment conditions, and I want to test the hypothesis that they increase in such a way that the relative differences between them stay the same. Pearson correlation ...
0
votes
0answers
33 views

Second Opinion(s) Requested: Doing 6 A/B Tests on a website with confidence levels around .8-92 is a bad idea, right?

A consultant recently used 6 A/B tests to compare the effects of various modifications to a website. The website is designed to prompt visitors to respond with an action. As a "control" for each test, ...
1
vote
0answers
25 views

Correction of P value for correlation matrix

If one has to perform correlation matrix, correction for multiple testing is likely to be useful. If significant P value is taken as (0.05/number_of_variables) (e.g. for 50 variables, 0.05/50 = ...
2
votes
1answer
37 views

How to estimate probability of $\geq$ n type I error in multiple testing comparison?

My question is about the calculation of the probability of making $\geq n$ type I error when $p$ independent statistical tests are made. I can calculate the probability of $\geq 1$ type I error with ...
0
votes
0answers
14 views

Little mess with confidence intervals for Holm-Bonferroni

I'm a newcomer in the problem of multiple comparisons. I wonder how to correct confidence intervals for Holm-Bonferroni method? I know that for Bonferroni method we can just change the confidence ...
0
votes
0answers
23 views

Are the statistical words contrasts and post hoc synonymous?

I have implemented a linear mixed-effects model using lmer{} model in R and used glht to to ...
1
vote
0answers
16 views

Bonferroni adjustment according to which families?

We have 17 subjects in a crossover design of three groups: Challenge agent (CA) CA+ compound of interest (CI) CI - Our data proves this has no physiological effect, and is thus not used in ...
0
votes
0answers
8 views

How to determine values of coefficients for a comparsion using factorial design?

I think my problem is best answered by answering the following example but the more general the answer and explanation the better: Given 2 factors X and Y, x with 2 levels x1 x2 and Y with three ...
3
votes
1answer
27 views
1
vote
1answer
17 views

Bonferroni method for multiple comparisons with 1-way layout

My statistics book describes the Bonferroni method of doing multiple comparisons with a one-way layout as essentially performing a t-test with each pair of interest with a significance level scaled ...
0
votes
0answers
11 views

False discovery rate from non-parametric tests with small n

I have a bunch of paired readings of bacterial abundance, sampled from two different places in the same patient and divided by total bacterial cells found in each location. Thousands of species were ...
1
vote
1answer
23 views

Unequal sample size one way ANOVA

I want to compare the means of X between BMI groups, but have unequal sample sizes between my groups. ...
-2
votes
0answers
44 views

Dissertation-results section

basically i am testing 2 species of pigs, 2 pigs of the same species in one enclosure and another 2 in a different enclosure, My aim is to see if alterations to feeding regimes have an effect on an ...
9
votes
3answers
353 views

Confusion with false discovery rate and multiple testing

I have read this great paper by David Colquhoun: An investigation of the false discovery rate and the misinterpretation of p-values (2014). In essence, he explains why false discovery rate (FDR) can ...
0
votes
0answers
23 views

Optimal model/statistical test for my design?

I have a design with 1 between-subject factor and 2 within-subject factors as independent variables and 5 dependent variables (longitudinal accelartion, lateral acc., response time, first conscious ...
0
votes
0answers
23 views

Pairwise comparisons with covariates

I have a simple ANCOVA model such that outcome ~ sex + group with outcome being continuous, ...
0
votes
0answers
25 views

Kruskal-Wallis post hoc comparisons different results in different software used

I'm having some trouble analyzing my data because when I use StatSoft Statistica and run a Kruskal-Wallis, I get one set of values for multiple comparisons, but when the same analysis is run in XLSTAT ...
0
votes
0answers
15 views

Dunnett for comparing multiple means to a specific value

In text-book statistics, one can use the one-sample t-test for comparing a population mean to a specific value. What would be the correct way to conduct a Dunnett's procedure of comparing multiple ...
3
votes
0answers
14 views

Multiple comparisons of parameters from non-linear regression

Hei, I want to compare parameters a_i and b_i estimated by nonlinear regression (y=a_i*x/(b_i+x)) for different data-sets (let's say 8 different data sets). I have calculated the non-linear ...
1
vote
0answers
24 views

Multiple t tests or an ANOVA?

I have low anxiety and high anxiety participants (21 in high anxiety and 20 In low). I measured anxiety and then categorised Pts according to this. They completed two memory tasks. In one memory ...
0
votes
0answers
38 views

Significant Friedman's test vs. non-significant multiple comparisons

I am currently running a randomized, placebo-controlled, cross-over design for a new type of intervention. This intervention is compared to a placebo and another more conventional type of treatment. ...
0
votes
0answers
37 views

compare muliptle probabilities

I have built three decision tree model to predict the response for three offers (One model per offer). I want to find the best offer for each customer based on the predicted probabilities. All offers ...
4
votes
2answers
162 views

Using multiple comparisons corrections in Fisher p-value framework

I am wary of stepping into the civil war between Fisher vs Neyman-Pearson interpretations of $p$-value (which has been well elucidated here and here), but I've been pondering a question that I keep ...
0
votes
0answers
47 views

P-values adjustment with known results of some hypotheses

I ran an experiment - within-subjects design; 2 scenarios (congruent and incongruent one). In each scenario I measured overall task completion time, geo. mean of task completion time and accuracy. ...
0
votes
0answers
19 views

Multiple testing confusion, what is significant?

I'm learning multiple hypothesis testing and this confuses the hell out of me. This is just a simple "are these distribution significantly different?" question. Data: Treatment: 100k random ...
0
votes
0answers
19 views

Which means difference test to use?

Currently, my data is count data and I am analyzing it in Proc Glimmix in SAS. My data focuses on how many times birds from two treatments performed stand, sit, forage, ect.... So it basically looks ...
0
votes
1answer
60 views

How to calculate critical values for Dunnett procedure given alpha, df1 and df2

I am looking for an R function that can calculate critical values for a Dunnett multiple comparison procedure for any family-wise error rate and degrees of freedoms. In short, I need similar ...
1
vote
0answers
37 views

Independent t test significant but post-hoc planned analysis not

I was analysing two sets of data. In most cases for the study post hoc planned comparisons identified significance (<0.05), however for one sample, this was not achieved. However an independent t ...
1
vote
0answers
63 views

How much confidence can we have in statistically-based ranking for “safe” international airlines? [closed]

Some cites for such ratings exist, but they are not meant to be predictive. For example AirlineRatings.com awards stars based on questions like Is the airline on the European Union (EU) Blacklist? ...
1
vote
1answer
17 views

How to Compare more than two Averages of Proportions or Means of Proportions

My experiment deals with $60$ subjects, one third of them belong to type A subjects, $i=1..20$, another third to type B, $i=21…40$ and finally type C, $i=41..60$. Each of these $60$ subjects from the ...
2
votes
2answers
77 views

Is it appropriate to do an ANOVA on a feature selected via inspecting PCA results?

I've been given a dataset consisting of 8 dimensional feature vectors for 4 classes of objects. I was asked to find the features that best distinguish the classes, and write up a short report. My ...
0
votes
0answers
16 views

Multiple comparisons with genetic * environment interactions

I am analysing the next models with logistic or linear regression: Y = G*E + confounders, where G is a genetic variable (SNP) and E is an environment variable. The problem is the next: I have 13 SNPs ...
7
votes
1answer
109 views

Plain language meaning of “dependent” and “independent” tests in the multiple comparisons literature?

In both the family-wise error rate (FWER) and false discovery rate (FDR) literature, particular methods of controlling FWER or FDR are said to be appropriate to dependent or independent tests. For ...
2
votes
2answers
75 views

Creating groups using two continuous variables without using median-splitting?

I have two continuous variables, one of individuals' retrospective childhood anxiety and another regarding their current level of anxiety. Research has demonstrated that during a snapshot of ...
0
votes
1answer
41 views

Multiple T Tests

On one of my questionnaires, which measure learning behaviours, after PCA, I have 4 subgroups, Efficacy/Perseverance/Effort/Achievement - I have run a T-Test and I have 4 sets of data comparing my ...
1
vote
0answers
44 views

Running 2x2x2 ANCOVA in SPSS

I need to conduct an ANCOVA on my data. While I've taken a number of statistics courses, I have never come across ANCOVAs and I want to get some feedback on the correct way to conduct this analysis. ...
0
votes
0answers
21 views

glht on gamm results

I have a GAMM model that has both an LME and a GAM part. I have used glht (multcomp) to check significant differences through the code: ...
0
votes
0answers
9 views

Fmax permutation for multiple comparisons [duplicate]

In an EEG experiment we presented stimuli from four conditions. This was a within-subject design. Dependent variable was the amplitude of brain activity. For each pre-defined regions of electrodes on ...
1
vote
1answer
45 views

simulating a multiple comparisons problem using R and bonferroni correction

I am actually working on an exercise for my students dealing with multiple comparisons. However, I was having trouble trying to simulate some data in R to demonstrate the problem with multiple ...
0
votes
0answers
67 views

Edited: Glmer model won't run all factor levels due to multicolinearity - can you have pseudo-multicolinearity?

I have a repeated measures design in which I have tested 112 individuals each with three different 20 second playback stimuli. I used a random block design where an individual was assigned to 1 of 6 ...
0
votes
1answer
53 views

Can assumptions of a statistical test be tested post-hoc?

Suppose I ran 100 1-factor ANOVAs. I find some significant results. Now, since ANOVA has some assumptions, I should test if they hold. So, can I run for example Levene's tests (to check equity of ...
0
votes
1answer
40 views

IBM SPSS QUESTION: Measure the perception of the respondents

My problem to solve with IBM SPSS is: I have got answers to a question about X topic that contains 15 options of multiple response and I want to characterize the perception of the correspondents ...
0
votes
1answer
32 views

How to compare multiple frequency distributions with some zeroes?

I have a data frame with frequencies for the teacher profession descriptors as it is perceived by 1st year, 2nd year and MA students. Some of this frequencies are zeroes. What statistical method can I ...
0
votes
1answer
116 views

Is it wise to use permutation test with many electrodes and many conditions?

I have data from an EEG (electrophysiological data) experiment where we have more than 10 electrodes recording brain activity (dependent variable) simultaneously. The design is a within subject design ...
0
votes
1answer
72 views

Post-tests to consider in mixed models

I have a mixed model of the form y~condition + Replicate + Error(Replicate) but at the moment, and am running into difficulties in performing a post-hoc test. I ...