Skip to main content

Questions tagged [multiplicity]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
22 views

Combining Survey Weights

I have an annual survey that is somewhat complex in design. Sample frames are pulled quarterly and overlap. Each quarter's sample is removed from the subsequent quarter's frame. Samples are stratified ...
ReliableResearch's user avatar
3 votes
1 answer
113 views

If we know that there were no type 1 errors in previous tests, should we still make corrections for multiplicity?

Suppose there's a study where an interim analysis was performed to evaluate efficacy with the intention of stopping the study. However, since the predefined success criterion wasn't achieved, the ...
user7064's user avatar
  • 2,207
2 votes
1 answer
46 views

Including Covariate P-Values in Multiple Comparison Corrections"

I'm currently working on a statistical analysis involving multiple comparisons and I'm uncertain about a specific aspect of the process. Specifically, my question is about the inclusion of p-values ...
Jbnimble's user avatar
1 vote
0 answers
110 views

Why are equally-wide confidence intervals used with the Benjamini-Hochberg correction?

This paper says: *2. For any p-values that are significant after FDR correction, construct a CI for the corresponding test with coverage $1-\alpha^{'}$, where $\alpha^{'}$ is: $\alpha^{'}=\frac{k}{m}\...
Federico Tedeschi's user avatar
0 votes
0 answers
47 views

Friedman test_multiplicity

I'm performing Friedman test to see if there is a difference between 8 treatments, then if it is significant I'll do Wilcoxon signed rank test to see which pair is significant. I wonder if I should ...
user365345's user avatar
4 votes
2 answers
67 views

Does the Prospective-Alpha-Allocation-Sequence (PAAS) actually control the family-wise error rate?

I recently read about the PAAS as an alternative to Bonferroni. Unlike Bonferroni, where $\alpha$ is subdivided among multiple tests such that $$ \alpha_1 + \alpha_2 + \ldots + \alpha_k = \alpha $$ ...
AdamO's user avatar
  • 63.4k
1 vote
1 answer
343 views

Equivalent to Dunnett's test for binomial endpoint?

I have a trial comparing 2 treatments vs a control group. Typically one would use Dunnett's test for this situation, but the outcome measure is binomial. It seems strange to me but I've looked for ...
Bosley's user avatar
  • 549
3 votes
3 answers
632 views

Are methods of p-value adjustment only necessary when a universal/overall hypothesis is the target of inference?

I am trying to understand more clearly in which cases an adjustment of p-values is necessary. At the moment my reasoning about this can be summarized as follows: Benjamini and Hochberg (1995) list ...
MrMax's user avatar
  • 226
0 votes
1 answer
30 views

Should multiplicity be addressed when doing preliminary examination of missingness?

When dealing with missing data, one recommended initial step is to examine the effect of the missingness in each variable on the other variables. This entails doing multiple tests where a significant ...
okobroko's user avatar
7 votes
1 answer
207 views

Possible multiplicity issues in equivalence testing

I'm currently looking to compare 3 groups, trying to prove equivalence between them. For this, 5 specific parameters are measured for each group, and a pairwise comparison for each parameter among the ...
Alex's user avatar
  • 71
0 votes
1 answer
424 views

P value adjustment for pairwise comparisons of negative binomial

Question 1) Am I appropriately doing the pairwise comparisons? Question 2) Why is the Sidak method used for the emmeans? Should I instead use adjust = "none" for these? Context: I have a ...
Tristan's user avatar
2 votes
0 answers
51 views

Multiplicity Correction for Coefficient Estimates in Linear Regression [duplicate]

When regressing a dependent variable y on some feature vector x with a standard linear regression, is there any correction in place for multiplicity or is this just not relevant in this case? The ...
aru_bdd's user avatar
  • 31
1 vote
1 answer
950 views

Multiplicity correction when using emtrends to see if slopes are different from 0

I have a relatively straightforward question but I haven't been able to find the answer. Consider this example: ...
user3640617's user avatar
8 votes
1 answer
307 views

Bayesian Inference in the presence of multiple hypotheses

"Because [Bayesian Inference] respects the forward flow of time or information, there's no need for nor availability of methods for correcting for multiplicity ... The evidence of one question ...
AdamO's user avatar
  • 63.4k