Linked Questions

72
votes
16answers
37k views

Practical thoughts on explanatory vs. predictive modeling

Back in April, I attended a talk at the UMD Math Department Statistics group seminar series called "To Explain or To Predict?". The talk was given by Prof. Galit Shmueli who teaches at UMD's Smith ...
18
votes
4answers
8k views

Misunderstanding a P-value?

So I've been reading a lot about how to correctly interpret a P-value, and from what I've read, the p-value says NOTHING about the probability that the null hypothesis is true or false. However, when ...
16
votes
3answers
6k views

What follows if we fail to reject the null hypothesis? [duplicate]

What conclusions can we draw if $p>\alpha$? Does not rejecting the $H_0$ mean anything?
7
votes
4answers
11k views

Many p-values are equal to 1 after Bonferroni correction; is it normal?

I have behavioral data for 10 animals that I studied at two different times of day, AM and PM. I am looking to see if there is a significant difference in 8 different behaviors at AM vs. PM. I have ...
7
votes
2answers
15k views

p-value correction for multiple t-tests?

My dataset consists of $n$ genes, each of them described by a vector of expression values, $5$ for "healthy" individuals, and $5$ for "unhealthy" individuals. I am going to run $n$ t-tests (one for ...
8
votes
3answers
4k views

Is there a consensus on adjusting alpha for multiple contrasts if the main effect is significant?

After talking to a couple of statisticians, reading some book sections, internet articles, and forums like this, I am still unclear about multiplicity adjustments of post hoc tests and contrasts. Let'...
8
votes
2answers
2k views

Is there a multiple testing problem when performing t-tests for multiple coeffcients in linear regression?

This question comes from a discussion on the recent post by @rvl It's all in the family; but do we include the in-laws too? Here's a common scenario that I've seen many times. A researcher runs a ...
1
vote
1answer
6k views

Implementing chi-square in python and testing on scipy's poisson and norm variates

I need to implement Pearson's chi-squared test to test random variates. But I get very different results using different sequence length,degrees of freedom or even seeds. Only a few times sequences ...
8
votes
2answers
613 views

Extent of multiple testing correction

A bit of a strange question. In my fourth year biostatistics class today, we were discussing when and when not to use multiple testing correction, and the professor made an offhand comment. He asked ...
4
votes
1answer
1k views

Controlling FWER using minP/maxT methods

A common approach for controlling the FWER in genomics analysis is the so-called minP/maxT procedure. See, for example: Dudoit, S., Shaffer, J.P., and Boldrick, J.C. (2003). Multiple Hypothesis ...
10
votes
2answers
517 views

What is the cause of the multiple comparisons problem?

I understand the intuition behind the MCP but I'm having trouble pinpointing exactly the cause, what is it that should be avoided, or at least accounted for. In its most blunt definition, I agree ...
2
votes
1answer
1k views

Correcting for multiple comparisons - same test on different data, vs different test on same data

I have a list of p-values and want to correct for multiple comparisons, i.e. control for the accumulation of false positives. In general, should I adjust my approach depending on whether the p-values ...
5
votes
1answer
367 views

Why bother looking at an omnibus ANOVA when I have a priori hypotheses about group differences?

I am examining three independent groups that were measured on a continuous outcome variable. I have a priori belief that the result should be Group 1 < Group 2 < Group 3. I've been told to do an ...
1
vote
1answer
480 views

qPCR Gene expression validation and multiple testing

It's common practice to adjust for multiple testing when analyzing high-throughput gene expression data, eg. RNA-seq and microarrays. Despite this, researchers routinely select a subset of genes for ...
0
votes
1answer
219 views

What's the problem with multiple comparison if you only test each hypothesis once?

I recently questioned my understanding of multiple comparison when I saw this xkcd comic: https://xkcd.com/882/. The comic is illustrating multiple comparison by showing a study that concludes that ...

15 30 50 per page