Hot answers tagged

9 votes

Does introduction of new variable always increase the p-val of existing ones?

@Dave explains that mathematically p-values can get bigger or smaller with more predictors because adding a predictor changes the model. But there is a fundamental problem with your argument: it is ...
user avatar
  • 4,352
8 votes
Accepted

Does introduction of new variable always increase the p-val of existing ones?

Definitely not, and there is a sense in which this is why we do regression. An example is analysis of covariance (ANCOVA), which is the usual ANOVA but with an additional variable that typically is ...
user avatar
  • 34k
5 votes

How can I manually calculate the Bonferroni correction?

No, unfortunately your correction formula is wrong. The Bonferroni correction compares a test statistic's $p$ value out of $m$ hypotheses to a nominal $\alpha$ by checking whether $$ p<\frac{\alpha}...
user avatar
4 votes

Statistical Signifance Test for discrete data

As an opening observation, you should never begin your statistical analysis with a goal of getting a particular result in your test --- rather than aiming to prove that the two sets of data come from ...
user avatar
  • 100k
3 votes
Accepted

False discovery rate correction when all P value are equal

The Benjamini-Hochberg FDR calculation performed by p.adjust seems perfectly appropriate in your situation. There is no theoretical problem with tied p-values in ...
user avatar
  • 10.2k
3 votes

Why is p-value produced by the K-S test decreasing with increased sample size even though samples are from the same distribution?

We expect a uniform distribution of p-values when the null hypothesis is true, so some of the p-values will be small. It’s just a bit of strange luck that the decrease happened to coincide with an ...
user avatar
  • 34k
2 votes

Is this ok to average p values?

It is quite common to assess statistical significance via resampling, bootstrap or permutation tests, all of which are randomized (see, e.g., the textbook by Good, 2006). In such cases, people will ...
user avatar
2 votes
Accepted

double-tail Bayesian "p value" à la MCMCglmm

You might find Reconnecting p-Value and Posterior Probability Under One- and Two-Sided Tests by Shi and Yin (2021) useful. They show interesting connections between the two-sided posterior probability ...
user avatar
  • 1,735
2 votes

Why does the sum of two metrics give us a p-value above significance level but each metric viewed separately has p-value below significance level?

I think one thing people are missing is that variance adds in quadrature. Without more information, it is impossible to say what the source of the problem is. But it could be possible that the ...
user avatar
1 vote

Negative Binomial - pvalues for log vs pvalues for response scale?

My preference with negative binomial glms is always to use likelihood ratio tests (LRTs) using R anova's function rather than Wald or least squares based tests. With LRTs there is no need to choose ...
user avatar
  • 10.2k
1 vote
Accepted

Statistical Signifance Test for discrete data

Assuming that you are treating the responses for Confidence as ordinal categories, your bar plot of counts for Confidence responses for each Group suggests that there is no systematic difference in ...
user avatar
1 vote

Why does the sum of two metrics give us a p-value above significance level but each metric viewed separately has p-value below significance level?

Possibly, and as a commenter was suggesting, there is some negative correlation between $X$ and $Y$. Another possibility is that they are correlated differently with the A and B groups themselves -- e....
user avatar
1 vote
Accepted

Not sure on how to interpret the p-value for a permutation test

I see two possibilities, of which one has been ruled out by OP in comments. The p-value in the first plot is displayed incorrectly e.g. 1.00E-3 truncated badly could be displayed as 1.00. [OP says ...
user avatar
  • 13.2k
1 vote
Accepted

How is power of an experiment practically useful when p-value is low?

In this theoretical example, the $p$-value wouldn't correspond to the mean of the actual distribution of the test statistic (the green one in your schematic). Rather, the $p$-value would come from a ...
user avatar
  • 53.6k
1 vote

Is there an equivalent to a Forest Plot that does not use an Odds Ratio?

Just to clear up a possible mis-conception here first there is no requirement to have odds ratios to draw a forest plot you just need an effect size with its confidence interval. I think what you are ...
user avatar
  • 16.7k

Only top scored, non community-wiki answers of a minimum length are eligible