K-S Test on estimated parameters? 22 Million datapoints I know this question has been asked before, but I am having trouble with what I need to do in order to have valid results from KS test
I have ~22 Million data points. Does this mean the p-values are bogus anyways? can I still report the test statistic from estimated parameters? I am just trying to get a feel for which distribution best fits my data. Thanks
 A: The p-values are what they are. Your large sample size, however, will (correctly) flag tiny differences as statistically significant, even though the difference might be too small to be interesting.
The lesson here is that p-values should not be used to guide you toward differences that are large; that would be effect size. What p-values do is suggest that you found data that are surprising given your assumed null hypothesis and that you perhaps should doubt or reject the null hypothesis.
It is not lazy science or poor statistical practice to observe a statistically significant difference but decide that it is too small to warrant attention. (In fact, I might argue that to be good science because it suggests that you understand the needs of your field and don’t find it especially interesting if a trillion people managed to lose a gram of weight$^{\dagger}$ on a weight-loss plan, no matter how compelling the evidence (p-value) that the plan was indeed responsible for the weight loss.)
$^{\dagger}$Mass, I know...
