Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.

learn more… | top users | synonyms (2)

0
votes
0answers
5 views

Pairwise comparison among variance

I would greatly appreciate any help on this! I'm doing a research where I need to compare variances (or standard deviation) of several samples (9 samples). I did some research and found the "Bartlett ...
1
vote
0answers
11 views

Is the Scheffé test of contrasts the “best-case” for post-hoc tests?

I am reading through Explaining Psychological Statistics, 4th Edition and am stumped after the following passage: Scheffé (1953) understood that the best anyone can do when creating a complex ...
1
vote
1answer
9 views

Is alpha error inflation a problem with multiple ordered logistic regressions?

Following problem: I'm doing a survey where my independent variables are: (1) the perceived attractiveness of a product on a 7-Likert-scale and (2) the willingness to invest in such a product as a ...
2
votes
1answer
31 views

How do I avoid multiple t-tests (or is it ok to use multiple t tests here?)

I've searched around CV for a bit, and have not found a good answer as to what test to use instead of multiple t-tests in my particular situation. I am interested in comparing mean size (of fish) ...
1
vote
0answers
19 views

How to interpret and report significant glmmADMB interaction term?

I am analyzing a data set using glmmADMB and determining significance of variables using likelihood ratio tests to compare models with and without a given variable. ...
0
votes
1answer
29 views

How to test multiple comparisons of means of variables in R

I have a multiple N (>20) variables. I want to test $H_0$ that in at least one pair of them one variable have significantly higher values then the other. I know that I can perform a bunch of $\frac{...
1
vote
1answer
59 views

Multiple comparisons when examining the results of unrelated trials

Imagine that you have a data set that is like the one below. (The data is all made up and is not specific to the problem.) ...
1
vote
1answer
13 views

Should one adjust analyses if interim analysis is planned for an outcome other than the primary outcome?

It is widely accepted that analyses of clinical trials have to be adjusted if interim analyses are planned in order to decide if early termination is necessary due to evidence for superiority of one ...
3
votes
1answer
68 views

Beginner question on dice roll statistical test

I have 2 cohorts of 10,000 people. I give each person in cohort 1 a die $a$, and ask them to roll it. I record the values as a tally. I then repeat this process with cohort 2 and give them die $b$. I ...
0
votes
0answers
17 views

Multiple testing correction with Benjamini-Hochberg with NaN values in the results

I have a list of p-values, and I want to correct them for multiple testing using Benjamini-Hochberg procedure. However, the list contains NaN/NA values. I know that ...
0
votes
0answers
3 views

the Univariate F statistics for each variable have the same rank order as the structure coefficients on the first CV

I have done a one-way MANOVA with 2 groups of 15 sample units each and 10 variables measured on each sampling unit, followed by separate ANOVAs and a CDA. I find that the Univariate F statistics for ...
3
votes
1answer
19 views

Only interested in select pairwise comparisons - Do I need to run an ANOVA first?

I have data with two biological factors. The first factor has 2 levels (say, Alive vs. Dead) and the second factor has 80 levels (protein of interest). I'm only interested in whether the mean value ...
0
votes
0answers
12 views

Control for multiple comparisons in an exploratory study?

I have an exploratory study where I investigate the relationship between several predictor variables (>10) and a few outcome variables (4). The large number of variables is not random but conceptually ...
0
votes
0answers
20 views

Comparing Subgroups vs Overall Relative Differences across Treatment

I have two very large unpaired control / variant groups, and a dependent variable that is the success/failure of a binomial, with a relatively small effect size between the treatment groups. If I ...
0
votes
0answers
20 views

How to interpret non significant F-test for main effect (genotype), but significant differences in the means of the genotypes?

I am analyzing an experimental data set based on the trait values under drought condition. The experiment was carried out on three separate drought environments. First, I have done a single ...
0
votes
0answers
21 views

When to use FDR?

I have a problem with the concept of false discovery rate. Assuming I have a panel of 50 anticancer drugs for which their IC50 values are reported for 30 cell lines. On the other hand, I have a ...
0
votes
0answers
9 views

comparisons of means within grouping factor (after two-way ANOVA)

Let's say in each of ten very different schools, one half of students is offered free lessons in physics and the other is not. After the next examination phase, the physics skills of the two groups ...
0
votes
1answer
36 views

Is there a concern for multiple comparisons when ranking instead of using significance?

Let's say I have many ANOVA tests on the same data using different factor variables each time. Then, I want to rank the F-statistics for all of those tests. Notice that I'm not interested in ...
0
votes
2answers
40 views

How to summarize R-squared of several regressions, one per subject

I will explain my question, I have made a study and I have 10 regression (one for each subject). I have a significance for each regression, but in some subjects the value of R-squared is 0.5 and in ...
0
votes
0answers
35 views

Multiple testing & model selection using AICc

I have a situation very similar to this post. I am finding the best-fit mixed effects model among a set of 5 candidate models for 7 different dependent variables (y1 to y7) using the same dataset of ...
1
vote
0answers
34 views

How to specify a significance test for intercept and slope between multiple latent groups

I have done a Latent Profile Analysis on 34 symptom-distress-items at beginning of psychotherapy treatment. I came up with 5 latent groups. The entropy of the model was 0.779 The average Latent ...
1
vote
0answers
23 views

Determining sample size using power analysis in multiple testing context

I am currently trying to determine the required sample size for an experiment. As I want to use the statistical power concept (for a t-test) to determine the required sample size, I want to specify a ...
0
votes
0answers
4 views

Compare Percentages - Pt 2

Based on earlier responses, I have further developed my testing methodology. The attached, if I have understood and applied responses correctly, gives me a z-score in the last column. Is it now just ...
0
votes
0answers
10 views

Log-rank test in R for file bmt

I am trying to find if there is any difference between the groups in file bmt in R package KMsurv using log-rank test. I have set H0: no difference in hazard rates between groups, and H1: there is a ...
0
votes
0answers
13 views

Non-parametric paired data: several Wilcoxon signed-rank tests? Other?

Scenario: I've taken blood samples from a group of patients (n = 15) and measured the expression of one marker on three cell populations within each sample (transitional B, naive B, memory B). I'm ...
1
vote
1answer
48 views

What statistical test allows comparison of the 3rd quartiles between groups?

I am using r and have ~400 observations divided into 3 groups. Group A contains 199 observations, while B contains 173 and C contains 24. I have generated a boxplot of their corresponding measures ...
1
vote
0answers
21 views

Multiple testing correction in a graph

I am doing some statistical tests over a graph. Namely, each subject in my study has a graph (composed of nodes and edges), where each edge has two different values (let's call them F and S). I am ...
1
vote
0answers
34 views

Multiple Hypothesis Testing with Categorical Variables and Alternatives

Suppose we have a variable $Y_i$ which is a binary categorical variable, and we have a variables: $X_{i,1}$, $X_{i,2}$, ....$X_{i,n}$ ($i=1,2,....N$ where $N$ is sample size, and $n$ is the number of ...
0
votes
0answers
11 views

Analyzing groups across time points in GraphPad

I have a dataset consisting of measurements of the level of 28 biomarkers for each of 25 people at 7 time points (4900 measurements in all, minus a few blanks). Each person is in a certain group ...
0
votes
0answers
23 views

Pairwise test and random effect for Beta regression

I have percentage areas values from 4 treatments (25 replicates per treatment). I would like to compare these percentages. I am supposed to use a beta regression, because my response variable is a ...
0
votes
1answer
14 views

Checking if an Error Type Occurred After Running A Hypotheis Test - Split (A/B) Test

When doing Split (A/B) Testing, or really any other Statistical Hypothesis test for that matter, there is always some chance of committing a Type 1 or Type 2 error. I am wondering if there are any ...
0
votes
0answers
36 views

Calculating Tukey HSD for a subset of comparisons - can I do this?

A colleague has about 50 samples, each with 2 conditions (treated and non-treated). The data is continuous and normal, and there are 3 data points in each sample-treatment combination. They'd like to ...
0
votes
0answers
17 views

Plotting errorbars adjusted for multiple comparisons

I have run a 1 way basic ANOVA on 4 categorical variables. Afterwards, I ran multiple comparisons on all 4 groups (6 comparisons) & adjusted using the Tukey-Kramer method. This was all done in ...
0
votes
0answers
25 views

How to see if 2 variables affect another variable?

I have a list of stocks with their % change, along with 2 other variables (Price Difference and Volume Difference). I would like to know how (if any) the variables Price Difference and Volume ...
0
votes
0answers
23 views

Interpreting Tukey HSD p-value After ANOVA + FDR correction

I'm currently analyzing a dataset where I have gene expression pathway activation scores across three groups: two different treatments and one control (so, basically a matrix of the form pathways x ...
1
vote
0answers
9 views

How to calculate FDR

This is quite a basic question, however I am having trouble. I have 10 samples. I have calculated the p value for 500 discrete elements within each of the 10 samples. The 500 elements are common to ...
0
votes
0answers
16 views

Comparing algorithms: runtime vs accuracy

I am comparing several algorithms with respect to their runtime and accuracy on several datasets. I follow [1] and compute the average rank using the accuracy and apply the Friedman and Nemenyi tests. ...
0
votes
1answer
21 views

Multivariate Regression vs T-test: and implication for multiple comparisons

I have a data frame with different outcome measures (DV1:4) for participants some partcipant with additional IVs 1:2. ...
0
votes
0answers
15 views

multicomparison treatment against 2 controls

i'm new in "r" and in statistics. I have some data that i would like to analize. Just to give you a background of my experiment: 1) i have 4 treatments (ifferent concentration of drugs) 2) i have 2 ...
0
votes
0answers
11 views

How many samples do I need to reliably apply the Bradley-Terry model?

I would like to use the Bradley-Terry model in order to build a ranking between objects, starting from a pairwise comparison test. However, I have almost 50 objects to be compared and the comparison ...
0
votes
0answers
10 views

Which analysis do I use for this experimental design?

Seeking feedback from those with a better quantitative mind than my own! Currently preparing my doctoral research proposal. I want to determine the impact that various levels of technology (...
0
votes
0answers
7 views

Corrections needed when performing multiple analyses on the same database? [duplicate]

I have access to data from various health-related surveys. Colleagues with a hypothesis in mind ask me to run their analyses. Even though each of them is unaware of the others' work, we are doing ...
0
votes
0answers
11 views

Multi-Model Inference - Post Hoc tests

Is there a way to do post hoc multicomparisons with Averaged Models of class "Average" gained from model.avg() of the MuMIn package in R?
2
votes
0answers
15 views

Degrees of freedom of a multiple comparission using estimability and lsmeans

I have a doubt about the degrees of freedom of a multiple comparission test. My data have the following structure: One data for each day, for 14 days, in each year season (auttum, winter, spring and ...
1
vote
1answer
22 views

When is it ok to NOT control for multiple comparison?

My initial feeling was never. But I notice that whenever people write up a larger study (or a meta-study) they do not adjust the significance thresholds of the individual measurements to account for ...
20
votes
3answers
2k views

Why aren't multiple hypothesis corrections applied to all experiments since the dawn of time?

We know that we must apply Benjamini Hochberg-like corrections for multiple hypothesis testing to experiments based on a single data set, in order to control the false discovery rate, otherwise all ...
0
votes
0answers
17 views

When to use MCP, or when to use closed testing procedure?

I am working as a programmer in a CRO and the pharmaceutical company tells us to carry out MCP test with 5 different contrasts. But I am not sure rationality of doing that without prior ANOVA F-test(...
0
votes
0answers
32 views

Statistical method to combine/summarize multiple R squared values

I have a set of measurements using two different methods. I Have 6 samples that I have measurements for using both methods. I am trying to demonstrate how comparable the measurements are from the ...
2
votes
2answers
79 views

Incremental adjustments of $\alpha$ for multiple hypothesis testing?

Given a nominal significance level of $\alpha$ for a single hypothesis test, I understand that if you do multiple tests, $i = 1$ to $m$, then according to Bonferroni's correction, the significance ...
0
votes
1answer
37 views

Assumptions on Multiple Comparison - Bonferroni and others

I was asked for an alternative procedure to multiple comparisons which does not require independence of the multiple tests being applied. I´ve found some readings where Bonferroni´s correction is said ...