Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.

learn more… | top users | synonyms (2)

0
votes
0answers
6 views

Statistical significance of difference between samples given total success rate when test was applied by group

I am comparing treatments that select members to receive some email, and we are measuring the response to that email. The emails are similar, but are sent to a different group of members and a ...
0
votes
0answers
23 views

Comparison of 2 data sets with different sample sizes

I have two data sets that have different sample sizes and I am unsure on how to compare them. Dataset 1 has 51 participants but dataset 2 has only 20 participants. The answers to the questions are ...
0
votes
0answers
20 views

Comparing similarities and differences of time series data

how to compare the shape of two time series data. E.g. comparing fluctuations of two time series data. like, how to quantify whether one is more fluctuating than the other?
3
votes
0answers
61 views
+100

Multiple comparisons after Kruskal Wallis using the FDR approach. How to compute P values (Dunn or Mann-Whitney)?

I want to allow users of GraphPad Prism to use the False Discovery Rate (FDR) approach to multiple comparisons after nonparametric one-way ANOVA (Kruskal-Wallis). The idea is first to compute an exact ...
1
vote
0answers
24 views

Why conduct large scale multiple comparisons rather than multiple regression?

Is there ever a compelling reason to conduct multiple hypothesis tests versus a multiple regression when comparing a response in two treatment groups among many (possibly thousands or millions) ...
0
votes
0answers
8 views

Analysis of pairwise differences in species composition between sampling dates

I have six timepoints (months) with 60 samples each. I want to assess the species composition changes over time. For single taxa i used glht() (multcomp) with the ...
0
votes
1answer
41 views

Multiple testing adjustment in linear models

I have troubles with multiple testing correction (For example, bonferroni adjustment. I'm aware that it's not most correct one). ...
2
votes
0answers
27 views

Paired Hypothesis Test in TWO Linear Regressions

I am wondering how to perform a paired hypothesis test in a panel regression. I have been looking in many stats books but I have not found anything. The question goes: assume for $i=1,2,\cdots, N$ ...
0
votes
0answers
18 views

Biostatistics: testing for differences in protein composition in two tissues measured longitudinally

Thank you for your help! I am confounded. I am trying to analyze experimental research data that looks like this: Dependent variable (DV) components: Two populations of cells (GFP and LacZ). Each ...
0
votes
0answers
23 views

Multiple testing correction for 2x2 tables in SPSS

Hi I have 1 binary dependent variable (presence or absence of coeliac disease) and multiple binary independent variables (presence or absence of different symptoms). If I perform a chi square in SPSS ...
0
votes
0answers
19 views

Is it statistically admissible to run multiple iterations of the same model but change the base level factor in order to get specific comparisons?

I have an agricultural experiment in which I was interested in seeing how parental identity of dam and sire impacted the number of buds formed and the resulting number of buds that survived ...
2
votes
0answers
29 views

Methods for meta-analysis: z-transform (Stouffer's approach) vs. Fisher's method vs… infinite others? [duplicate]

Fisher's method and Stouffer's approach (z-transform) for meta-analysis roughly follow the same scheme: take the p-values from some different experiments on the same hypotheses apply to them some ...
3
votes
1answer
129 views

Statistical significance test for averages of correlation coefficients

My setting looks like that I have several subjects $i$ (let's say 5) and for each subject I measure the lists/vectors $A_i$, $B_i$, $C_i$ and $D_i$. Then, for each subject $i$, I calculate the ...
0
votes
0answers
6 views

Correcting error for multiple correlations

I have been given a dataset to look at, which is made up of lots of clinical assessments. We have one main measure we are interested in (looking at sleep), which has 9 subscales. Then there are a ...
0
votes
0answers
42 views

Forcing multcomp:glht() to calculate as many meaningful pairwise factor combination as possible

I have a very imbalanced dataset consisting of several habitats (i.e. "woodland", "grassland", "swamp", and so on) with different sample sizes (10 - 40) and species counts as outcome. Species counts ...
0
votes
0answers
37 views

Post-hoc analyses with a synthetic control?

When running a traditional hypothesis test, it's common to "break out" the result by sub-groups using various post-hoc methods that control for multiple comparisons, such as the False Discovery Rate. ...
4
votes
2answers
53 views

Best way to test that one mean is greater than all the others

Suppose I have $k$ samples of different sizes, each of a different univariate variable. I want to test the significance of the hypothesis that the population of $k_0$ has a mean greater than all of ...
0
votes
0answers
22 views

Bonferroni correction for multiple testing and overlapping groups

I have allele frequency observations from a large geographic region that can be logically subdivided into smaller areas in a number of different and meaningful ways. If I were to test these ...
3
votes
1answer
36 views

Possibility to run multiple chi-square tests on 2x5 table

I'm analyzing data in which I have a dependent variable with two possible outcomes (yes or no) and an independent variable with five emotion conditions, neutral, angry, sad, ashamed, and afraid. The ...
1
vote
0answers
15 views

fdr correction for omics data

I have a experiment design with 2 factors and I have 850 genes from this experiment. I want to look at individual genes before i go to multivariate dimension reduction . So I plan to fit a two way ...
0
votes
0answers
16 views

Multiple Comparisons for paired control and treatment groups

I want to conduct a post-hoc analysis on the impact of a treatment compared to the control Each control and treatment were selected to that the underline features are similar. For e.g. in the ...
0
votes
1answer
33 views

Pairwise non-parametric test for small sample sizes

I was asked to check a small dataset (3 replicates for 5 treatments, each) for significant outcome differences. I give an example: The variance within replicates in the data cannot be assumed ...
2
votes
1answer
25 views

Does the bonferonni correction change the outcome of the p-value or is it just changin what you accept as significant?

For some reason I can't seem to integrate the bonferonni corrected p adjustment into my multiple comparisons t test in r so I need to do each of my 3 tests individually. What I wondered was do I need ...
0
votes
0answers
24 views

How to correct for multiple post-hoc tests in multi-factor ANOVA

Let's for the sake of this question assume we did an ANOVA with three factors A (3 levels), B (4 levels), and C (5 levels). We get significant main effects for A and B. In consequence we do three ...
0
votes
0answers
19 views

Inquiry on comparing quadratic regression models

I am conducting a project in which I am analyzing the relationship between stream flow and algal biomass for a 28 day interval. I have 5 different flow treatments (ranging from low to high flows). I ...
0
votes
0answers
21 views

Multiple Comparison Correction for Linear Regressions with Dependent Variables

I could use some help clarifying the use of multiple comparison corrections for a series of linear regressions when some of the variables being tested are calculated from each other. Toy problem to ...
5
votes
3answers
146 views

Should I adjust p-values when investigating an ANOVA interaction?

I'm conducting an experiment to compare 4 types of headphones. I'm interested in finding out whether the headphones differ in their subjective quality. Thirty subjects listened to each pair of ...
0
votes
0answers
54 views

post hoc test for unequal sample size and variance

I have a data frame on which I am doing ANOVA using three fixed independent factors of which one is gender (two levels 'male' and 'female'), temperature (three levels: 1,2,3) and quality (two levels: ...
0
votes
2answers
33 views

Increasing the power by dropping points: can I do it?

I am repeating a test on a large amount of data and FDR-correcting the p-values afterwards for multiple testing. Yet I still do not have enough power. However, I feel like it is not necessary to test ...
2
votes
0answers
26 views

Multiple comparison of non-normal, heteroscedastic data. What test should I use?

I have a set of brain pathology data. These were obtained by counting certain parameters in the brain. Due to availability of human brains, the amount of cases vary a lot across the different groups ...
0
votes
1answer
28 views

(Non-parametric) test for similarity between three or more vectors of observations?

I have three vectors each of six observations. I imagine other situations will arrive in which I might have more than three vectors with other lengths. I want a statistical test to compare them. ...
0
votes
0answers
14 views

TukeyHSD for non-pairwise comparisons

Following this question: Pairwise comparisons I'm not sure if I can perform a Tukey Honest Significant Difference test on the data I have. I've thought Tukey's method was only valid for pairwise ...
1
vote
2answers
38 views

How to adjust for multiple comparisons with 2 groups where several outcomes were measured?

I have two groups, Controls and Patients, with 6 measurements for each such as speed, amplitude and so on. I'd like to examine whether these measures are different for the 2 groups. So firstly I'd ...
0
votes
1answer
44 views

two-way multiple comparisons in R

This is a subset of my dataset: ...
1
vote
0answers
36 views

Generating t statistics from p values - binomial test multiple comparisons

I am using the binomial test to generate a series of p values. However, my method of multiple comparisons correction that I wish to apply to this series requires that I also have the corresponding t ...
0
votes
1answer
35 views

Confusing LSMEANS results

I am having trouble interpreting results from a multiple comparison test. Here is the model I ran: ...
3
votes
3answers
70 views

How to find the right test?

I have a problem in medicine: Each patient can have several (3) symptoms (A,B,C) of a disease at the same time. For each symptomtype we can measure a score i.e. A=0.2, B=0.45, C=0.8. The aim is that ...
0
votes
1answer
114 views

How to compare stochastic algorithms, before and after modification

I would appreciate suggestions on how I could make some specific statistical claims about the efficacy of changes to an algorithm. My statistics background is pretty minimal (I'm working on it). I ...
0
votes
0answers
40 views

Need a method for determining variable groupings in R

I am using R and trying to group one of my variables into larger groups so they have credibility. I have been manually setting each factor of the variable as the reference level, looking at all other ...
1
vote
1answer
47 views

Adjustment of p-values for multiple-comparisons in lsmeans

Using R's lsmeans, I'm testing the pairwise comparisons among factor1 levels for each different combination of factor2 and factor3 levels: ...
1
vote
0answers
23 views

How to determine statistical differences among locations and years for continuous (non-normal) temperature data using R

I have three years of daily water temperature data from four different rivers. The temperature data are not normally distributed, so I assume I need to use a non-parametric test. I would like to ...
1
vote
1answer
44 views

What is the difference between nominal significance and genome wide significance?

I need to determine a reasonable sample size for my Inifinium 450k methylation array case-control experiment and wondered why epigenome-wide association scan (EWAS) studies typically use a P value ...
1
vote
2answers
52 views

Comparing prevalence among sites. Fisher.test and pairwise.t.test

I'm studying the prevalence of one parasite on a host from different localities. I do it by assigning presence ("1") or absence ("0") to each host sampled. After the sampling, I got something like ...
3
votes
0answers
19 views

The link between FWER and max/min statistic

I am currently looking at lecture slides about fMRI data and I have the following: Let $H_{0_i}$ be the hypothesis that there is no activation in voxel $i$ (a cube area in the brain, a 3d version of ...
3
votes
2answers
62 views

What exactly is multiple testing / Bonferroni Correction? Can I use it with a single AB test resulting in a p-value?

I'm having trouble drawing the link between a single hypothesis test and when you would apply the Bonferroni Correction to apply for multiple testing. The way I see it, you effectively just ...
1
vote
1answer
68 views

R, Pairwise comparison of random variable in mixed model

We measured temperatures of a pond repeatedly every day at each hour for a month at two different depths (i.e., top and bottom). We want to see if the temperatures at the top of the pond are ...
2
votes
0answers
51 views

MANOVA as protection against Type I errors

OK, so I have absolutely scoured the internet, and it appears that there is consensus (among people who really care about Type I error) that doing a MANOVA before doing multiple ANOVAs does nothing to ...
0
votes
0answers
15 views

Is there an accepted way to compare within all demographic categories (e.g., within males, within females, within whites, etc.) using MANOVA

I have three experimental groups (group A, group B, and group C), and gave all participants the same 50-question questionnaire. I then can run a MANOVA to determine if their responses (quantitative) ...
3
votes
1answer
41 views

Why does power reduce when increasing the number of post-hoc comparisons?

I'm performing a Dunn's post test comparison of my data after a Kruskal-Wallis test (4 groups, treated and control each one has drug and vehicle injection). The comparisons I've been doing are ...
0
votes
0answers
24 views

What regression analysis to use? IVs with two levels and a DV with two conditions?

I'm trying to figure out what the best regression test to use for my data. I have three predictor IVs each with two levels. I also have a DV values belonging to two different conditions (A & B). ...