Linked Questions

16
votes
1answer
41k views

sample size too large? [duplicate]

I always thought larger sample sizes were better. Then I read something somewhere about how when sample sizes are larger, it's easier to find significant p-values when they're not really there (i.e., ...
2
votes
1answer
1k views

Problem with Mann-Whitney U-test for large samples [duplicate]

I am comparing two categories (A and B) of social media posts with the corresponding number of likes for each one. I am taking samples of equal size from each category at random and performing a Mann-...
1
vote
1answer
422 views

What to do with a very big sample size? (effect size) [duplicate]

We have a peculiar problem, for which we have many questions, but cannot seem to find satisfactory answers (we did find a few ideas as to what to do, but we are not very certain they apply to our ...
0
votes
1answer
89 views

Is it possible to have too much data? When should I subsample? [duplicate]

Social science PhD student here looking for a little help. I set out to collect around 300 online surveys for a study, but ended up with over 4,000. At that sample size should I sub-sample my ...
0
votes
1answer
85 views

Question regarding significance level [duplicate]

Say, for instance, I'm estimating a model with about 5 million observations using linear regression or MLE. Given that the estimates are consistent, using the standard rule of rejecting the null on a ...
0
votes
0answers
29 views

Is a test with small effect size and high sensitivity meaningful or useful? [duplicate]

From a reply by John What is true is that trivially small effects can be found with very large sample sizes. That does not suggest that you shouldn't have such large sample sizes. What it means is ...
298
votes
16answers
101k views

Is normality testing 'essentially useless'?

A former colleague once argued to me as follows: We usually apply normality tests to the results of processes that, under the null, generate random variables that are only asymptotically or ...
98
votes
9answers
14k views

Is this really how p-values work? Can a million research papers per year be based on pure randomness?

I'm very new to statistics, and I'm just learning to understand the basics, including $p$-values. But there is a huge question mark in my mind right now, and I kind of hope my understanding is wrong. ...
107
votes
5answers
18k views

What skills are required to perform large scale statistical analyses?

Many statistical jobs ask for experience with large scale data. What are the sorts of statistical and computational skills that would be need for working with large data sets. For example, how about ...
9
votes
2answers
4k views

Why are 0.05 < p < 0.95 results called false positives?

Edit: The basis of my question is flawed, and I need to spend some time figuring out whether it can even be made to make sense. Edit 2: Clarifying that I recognize that a p-value isn't a direct ...
6
votes
2answers
768 views

Why will a statistic be significant with sufficiently large samples unless the population effect is exactly zero?

From Wikipedia Given a sufficiently large sample size, a statistical comparison will always show a significant difference unless the population effect size is exactly zero. For example, a ...
4
votes
3answers
1k views

Real-world example on significance testing with large samples

The issues that arise when one uses statistical hypothesis testing framework with large samples are dissussed on CV (see e.g. this thread). The main problem discussed there is that in the real world ...
3
votes
2answers
1k views

Understanding the direction & strength of a correlation, & causal status, from a chi-squared test

I am analyzing Stack Overflow Posts. So I have a database with 1000000 questions, their current score (upvote or downvote) and a flag, that there is a source code part in the question (or not). So I ...
3
votes
2answers
1k views

How to deal with correlations not statistically significantly different from zero?

I often find parts of financial correlation matrices not statistically significantly different from zero. Sometimes, these correlations have a tangible effect on results - low correlations lead to ...
3
votes
1answer
3k views

Why does a large sample size cause a significant ANOVA F-test?

If I have a large sample size, e.g. 100,000 data points, I know that most significance tests are going to come back with a very small p-value unless the null hypothesis is "true on the nose." In other ...

15 30 50 per page