9
votes
Does introduction of new variable always increase the p-val of existing ones?
@Dave explains that mathematically p-values can get bigger or smaller with more predictors because adding a predictor changes the model.
But there is a fundamental problem with your argument: it is ...
- 4,352
8
votes
Accepted
Does introduction of new variable always increase the p-val of existing ones?
Definitely not, and there is a sense in which this is why we do regression.
An example is analysis of covariance (ANCOVA), which is the usual ANOVA but with an additional variable that typically is ...
- 34k
5
votes
How can I manually calculate the Bonferroni correction?
No, unfortunately your correction formula is wrong.
The Bonferroni correction compares a test statistic's $p$ value out of $m$ hypotheses to a nominal $\alpha$ by checking whether
$$ p<\frac{\alpha}...
- 99.7k
4
votes
Statistical Signifance Test for discrete data
As an opening observation, you should never begin your statistical analysis with a goal of getting a particular result in your test --- rather than aiming to prove that the two sets of data come from ...
- 100k
3
votes
Accepted
False discovery rate correction when all P value are equal
The Benjamini-Hochberg FDR calculation performed by p.adjust seems perfectly appropriate in your situation. There is no theoretical problem with tied p-values in ...
- 10.2k
3
votes
Why is p-value produced by the K-S test decreasing with increased sample size even though samples are from the same distribution?
We expect a uniform distribution of p-values when the null hypothesis is true, so some of the p-values will be small. It’s just a bit of strange luck that the decrease happened to coincide with an ...
- 34k
2
votes
Is this ok to average p values?
It is quite common to assess statistical significance via resampling, bootstrap or permutation tests, all of which are randomized (see, e.g., the textbook by Good, 2006). In such cases, people will ...
- 99.7k
2
votes
Accepted
double-tail Bayesian "p value" à la MCMCglmm
You might find Reconnecting p-Value and Posterior Probability Under One- and Two-Sided Tests by Shi and Yin (2021) useful.
They show interesting connections between the two-sided posterior probability ...
- 1,735
2
votes
Why does the sum of two metrics give us a p-value above significance level but each metric viewed separately has p-value below significance level?
I think one thing people are missing is that variance adds in quadrature. Without more information, it is impossible to say what the source of the problem is. But it could be possible that the ...
- 27.1k
1
vote
Negative Binomial - pvalues for log vs pvalues for response scale?
My preference with negative binomial glms is always to use likelihood ratio tests (LRTs) using R anova's function rather than Wald or least squares based tests. With LRTs there is no need to choose ...
- 10.2k
1
vote
Accepted
Statistical Signifance Test for discrete data
Assuming that you are treating the responses for Confidence as ordinal categories, your bar plot of counts for Confidence responses for each Group suggests that there is no systematic difference in ...
- 7,763
1
vote
Why does the sum of two metrics give us a p-value above significance level but each metric viewed separately has p-value below significance level?
Possibly, and as a commenter was suggesting, there is some negative correlation between $X$ and $Y$. Another possibility is that they are correlated differently with the A and B groups themselves -- e....
- 61
1
vote
Accepted
Not sure on how to interpret the p-value for a permutation test
I see two possibilities, of which one has been ruled out by OP in comments.
The p-value in the first plot is displayed incorrectly e.g. 1.00E-3 truncated badly could be displayed as 1.00. [OP says ...
- 13.2k
1
vote
Accepted
How is power of an experiment practically useful when p-value is low?
In this theoretical example, the $p$-value wouldn't correspond to the mean of the actual distribution of the test statistic (the green one in your schematic). Rather, the $p$-value would come from a ...
- 53.6k
1
vote
Is there an equivalent to a Forest Plot that does not use an Odds Ratio?
Just to clear up a possible mis-conception here first there is no requirement to have odds ratios to draw a forest plot you just need an effect size with its confidence interval.
I think what you are ...
- 16.7k
Only top scored, non community-wiki answers of a minimum length are eligible