14 votes
Accepted

interpretation of p-value near alpha level

There are two different approaches to interpreting statistical significance - the Fisher way, and the Neyman-Pearson way. We smush these together (into what Gerd Gigerenzer has called a 'bastardised ...
user avatar
  • 14.3k
7 votes

How to report a P-value?

When is it appropriate to give the exact P-value, instead of writing e.g. P<0.05 (also in case of non-significant P-values)? As a general guideline you want to convey as much information about ...
user avatar
5 votes
Accepted

Unusally high F value ANOVA

With sample sizes as large as those you will almost always find that the statistical test gives very low p-values. Divide those SD values by the square root of the sample sizes to see the estimated SD ...
user avatar
  • 11.2k
3 votes
Accepted

Usefulness of KS tests and other similar distribution comparing tests

This is one of those ideas that starts out sounding great but winds up being less helpful than one might hope. For instance, just because KS (or a similar test) says that the feature has a different ...
user avatar
  • 31k
2 votes
Accepted

Statistical test to check if data come from two different lines or just one

This is a standard application of an interaction term in a model. With x as continuous-valued predictors and y as your outcomes, let S represent which set a data ...
user avatar
  • 62.2k
1 vote

Permutation testing: Which variable should be shuffled?

With respect to a simple correlation between a single X and a single Y, you could shuffle either. That said, the p-value isn't "how often the resulting correlation coefficient exceeds the ...
user avatar
1 vote
Accepted

How many values should one validate to achieve a desired confidence?

There are a few ways to think about this. If you want to use the confidence interval approach, then you should also look at the "finite population correction" (Googling for that phrase ...
user avatar
  • 46.9k
1 vote

Is confounding a source of Type I errors?

Confounding can result in the identification of spurious, non-causal relationships that you don't care about, but I would not characterize these as Type I errors since there is a real statistical ...
user avatar
1 vote

How to report a P-value?

In practice, people report the exact p-value then declare whether the hypothesis test is statistically significant. Most statisticians don't advocate this. if the goal is hypothesis testing then ...
user avatar
  • 52.9k
1 vote

Which statistics test should I use?

Comment: A suggestion has been made to do a chi-squared test on your $2 \times 6$ table of counts to see if the distributions for High and Low are different. That is one reasonable possibility among ...
user avatar
  • 49.7k
1 vote
Accepted

How do I test that two samples don’t differ significantly by their demographics?

You can run tests to check that demographic variables are balanced. However, it's not recommended to do so for a randomized study. As you say, you have to trust the randomization. What's recommended ...
user avatar
  • 2,156
1 vote

Wilcoxon.test in R will not calculate exact distribution due to ties (scipy.stats.wilcoxon will)

I don't have enough reputation points to add this as a comment, so although not an answer, hope it does help. I've adjusted your code a bit for convenience: ...
user avatar
  • 11

Only top scored, non community-wiki answers of a minimum length are eligible