Timeline for Probability that Null Hypothesis is True
Current License: CC BY-SA 3.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 6, 2019 at 13:55 | comment | added | BigBendRegion | A lot of the confusion here would be clarified if we abandon the finite population reference. Finite populations are not the scientifically interesting targets of inference. Say the even SSNs have higher mean weight in some existing population than the odd SSNs. Is that a scientifically interesting fact? Does it generalize? Is there some causal explanation? Obviously no, no, and no. Except in demography studies, finite populations are not interesting as targets of inference. Replace "population" with "data generating process," and then you have something that is scientifically interesting. | |
| Sep 29, 2017 at 16:21 | comment | added | emory | @gung I implicitly took the population as the set of people who currently have SSN and you are taking the population as the set of people who have ever had or ever will have SSN. Even then the population is still finite - much less than 2^64. The weigh ins will take much longer and pose some interesting research questions - like at what life point do we weigh individuals - birth, SSN issuance, or death; and how to obtain weights for the deceased. The measurements will be of no practical value b/c they will not be available until after our present society has radically changed. | |
| Sep 29, 2017 at 14:08 | comment | added | gung - Reinstate Monica | @emory, that's not the population, that's a sample from an infinite theoretical population. We don't ultimately care only about these exact people, but anyone who could be assigned an SSN. In theory, we could get measures of everyone today & find weights for evens are +1x10^-200, wait 10 years (so that there's a slightly different set of existing people) & find that weights of evens are -1x10^-200. Do we believe that even SSNs have both a positive and a negative effect on weight? That isn't what people are really trying to infer & neither would tell us if there is a direct relationship. | |
| Sep 29, 2017 at 13:57 | comment | added | emory | @gung the population - the set of people who have ever been assigned a social security number and are still alive - is very large but definitely finite. It is in theory possible to weigh them all and produce descriptive (not inferential) population statistics. We commonly think of it as infinite b/c it is easier computationally and the difference is negligible, but it is not. | |
| Sep 29, 2017 at 13:40 | comment | added | Sextus Empiricus | ...Or in other words. We use a sample to estimate a population and compare the population with theory. The observed population may be "proof" (e.g. p>0.05 as in Peter's example) that the population is different from zero. But the observation of the population being different from zero (with high precision) is not necessarily strong proof that the theory is wrong, and actually it is very likely that we do not observe the mean value of some model P(population mean = theoretic mean) = 0 (in many cases). And, the error of the population estimate, is not the error that the theory allows. | |
| Sep 29, 2017 at 13:30 | comment | added | Sextus Empiricus | Gung, If the sample size approaches the population size then the SEM becomes incredibly small and we will see, with high probability, a deviation from some theoretic thought. This can be understood by the population itself being a sample from a theoretic distribution. And at this point my thought comes along (which was more like addressing the comments from @emore, or at least, I was trying to fill a gap that I may not have understood), it is very likely that a population is different from a theoretic $H_0$, because these hypotheses are often defined as a point rather than an interval. | |
| Sep 29, 2017 at 13:00 | comment | added | gung - Reinstate Monica | @MartijnWeterings, I don't follow your argument. Are you trying to prove that there has to be a direct relationship b/t all possible variables from first principles? What is "X" supposed to be here? There either is a relationship, or there isn't. The probability can only be either $1$ or $0$. I don't see what the density of a normal distribution has to do with that, but then, I don't follow your argument & you may not even be trying to address my comments. | |
| Sep 29, 2017 at 12:53 | comment | added | gung - Reinstate Monica | @emory, so you "know" this because you proved it in a hypothetical study? That's begging the question. The population, for standard statistical inference, is infinite; it's all the people who could be under the given conditions, not the people who happen to be alive today. The question of rounding is also a red herring; weights vary be enough for that to be inconsequential & w/ enough data, an arbitrarily small difference could be resolved, even w/ rounding. I also can't tell if you understand my distinction about marginal associations. | |
| Sep 29, 2017 at 12:16 | comment | added | Peter Flom | @gung I suppose that, since individuals' weights are recorded to the nearest pound (or kg) you could get no effect at all in the population, it seems so unlikely that it can be considered impossible. Divide 300 million integers into two piles of 150 million each. Make the integers be rounded from a normal distribution if you like. There are a LOT of ways to do this and a vanishingly small number where the piles will be equal. | |
| Sep 29, 2017 at 12:12 | comment | added | Peter Flom | @KalevMaricq If it's just the prevalence of a disease, then you can use the prevalence. If it is something else (e.g. a logistic regression) you can use the effect size from whatever else it is. | |
| Sep 28, 2017 at 19:25 | comment | added | Sextus Empiricus | Say you have $X \sim \mathcal{N}(\mu,\sigma)$ then the density f(X) may be non-zero but $P(X=0) \equiv lim_{d->0} P( 0-d<X<0+d) = \int_0^0 f(X) dX =0$. If you allow marginal, then a (experimental) relationship/correlation between weight and SNN is very likely (still unlikely that it is a causal relationship, in any direction). | |
| Sep 28, 2017 at 15:57 | comment | added | emory | I know that weight is a continuous variable. Although we might record it as an integer number of kilograms. Your comment was about an observational study (drawing inferences about a population based on a sample). Since my study is funded by hypothetical dollars it is a population study using infinite precision scales - no need for statistical inference. | |
| Sep 28, 2017 at 15:44 | comment | added | gung - Reinstate Monica | That's surprising @emory. How do you know that? Has it been studied? Can you cite the paper? Are you referring to a direct relationship, or a marginal one? (Nb, I stated that there would be a marginal association.) | |
| Sep 28, 2017 at 15:34 | comment | added | emory | @gung you can believe whatever you want, but there is definitely a non-zero relationship between weight and SSN. We do know anything more about the relationship other than its existence and that it is probably small. | |
| Sep 28, 2017 at 14:53 | comment | added | gung - Reinstate Monica | FWIW, I'm perfectly willing to believe that there is no relationship between a person's weight & whether their SSN is odd or even. In an observational study, these variables will be correlated w/ some other variables, etc, such that there is ultimately a non-0 marginal association. I think the valid point is that, for most things researchers invest their time to investigate, there is some decent reason to suspect that there is real a non-0 effect. | |
| Sep 28, 2017 at 13:57 | comment | added | Kalev Maricq | Good point about effect size. Is there an analogue to situations like testing for a disease, where the question is Boolean in nature? | |
| Sep 28, 2017 at 10:57 | comment | added | Peter Flom | Well, we can never really know p(H0|data). Not with a small sample and not with a large sample and I don't think Bayesians really get around this, but I'm no expert on Bayesianism. But even with a small amount of data, effect size estimates are important. | |
| Sep 27, 2017 at 20:45 | comment | added | David Ernst | Not that I disagree with you, but don't you think when he worries about p(data|H0) or p(H0|data) he's talking about studies with low $n$. The example you give is easy in both frameworks bayesian and frequentist because their respective weaknesses/subjectivity don't matter in light of abundant data. The only error you can still make in this situation that would matter is to confuse significance with effect size. | |
| Sep 27, 2017 at 20:39 | history | answered | Peter Flom | CC BY-SA 3.0 |