14
votes
Accepted
interpretation of p-value near alpha level
There are two different approaches to interpreting statistical significance - the Fisher way, and the Neyman-Pearson way. We smush these together (into what Gerd Gigerenzer has called a 'bastardised ...
- 14.3k
7
votes
How to report a P-value?
When is it appropriate to give the exact P-value, instead of writing e.g. P<0.05 (also in case of non-significant P-values)?
As a general guideline you want to convey as much information about ...
- 46.4k
5
votes
Accepted
Unusally high F value ANOVA
With sample sizes as large as those you will almost always find that the statistical test gives very low p-values. Divide those SD values by the square root of the sample sizes to see the estimated SD ...
- 11.2k
3
votes
Accepted
Usefulness of KS tests and other similar distribution comparing tests
This is one of those ideas that starts out sounding great but winds up being less helpful than one might hope.
For instance, just because KS (or a similar test) says that the feature has a different ...
- 31k
2
votes
Accepted
Statistical test to check if data come from two different lines or just one
This is a standard application of an interaction term in a model.
With x as continuous-valued predictors and y as your outcomes, let S represent which set a data ...
- 62.2k
1
vote
Permutation testing: Which variable should be shuffled?
With respect to a simple correlation between a single X and a single Y, you could shuffle either. That said, the p-value isn't "how often the resulting correlation coefficient exceeds the ...
1
vote
Accepted
How many values should one validate to achieve a desired confidence?
There are a few ways to think about this.
If you want to use the confidence interval approach, then you should also look at the "finite population correction" (Googling for that phrase ...
- 46.9k
1
vote
Is confounding a source of Type I errors?
Confounding can result in the identification of spurious, non-causal relationships that you don't care about, but I would not characterize these as Type I errors since there is a real statistical ...
- 5,733
1
vote
How to report a P-value?
In practice, people report the exact p-value then declare whether the hypothesis test is statistically significant. Most statisticians don't advocate this. if the goal is hypothesis testing then ...
- 52.9k
1
vote
Which statistics test should I use?
Comment:
A suggestion has been made to do a chi-squared
test on your $2 \times 6$ table of counts to see if the distributions for High and Low are
different. That is one reasonable possibility among ...
- 49.7k
1
vote
Accepted
How do I test that two samples don’t differ significantly by their demographics?
You can run tests to check that demographic variables are balanced. However, it's not recommended to do so for a randomized study. As you say, you have to trust the randomization. What's recommended ...
- 2,156
1
vote
Wilcoxon.test in R will not calculate exact distribution due to ties (scipy.stats.wilcoxon will)
I don't have enough reputation points to add this as a comment, so although not an answer, hope it does help.
I've adjusted your code a bit for convenience:
...
- 11
Only top scored, non community-wiki answers of a minimum length are eligible