Timeline for Umbrella term for issues when not having enough data to properly do a test.
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 11, 2015 at 12:46 | history | edited | conjugateprior | CC BY-SA 3.0 |
spelling
|
| Sep 11, 2015 at 12:01 | comment | added | conjugateprior | I think @tim is correct that power is the right concept. Consider that, in addition to the model specification, power is a function of desired $\alpha$, effect size, and sample size. While it's traditional to fix model, $\alpha=0.05$ (or somesuch), and expected effect size, then compute a required sample size, the concept can be used other ways. Specifically, desired $\alpha$ is tightly connected to confidence interval width. So requiring a smaller $\alpha$ in the previous power computation indirectly demands a more precise i.e. narrower, intervals, which is what you want. | |
| Sep 11, 2015 at 11:12 | comment | added | JonB | Yes of course, but it is a situation in which the sample size might be too small to provide a reliable estimate of the effect of the independent variable of interest. So, not an issue of power but still a too small sample size. The issue might be a new and expensive drug, and we then want to determine the size of the effect, not just that it is better than the existing preferred treatment. The sample size is too small to give a reliable estimate, but it does not lack the power to detect a difference. So problem with a small sample size isn't equivalent to lack of power? | |
| Sep 11, 2015 at 10:53 | comment | added | Tim | @JonasBerge Then the test is able to correctly reject null but the confidence intervals are wide... Test works properly in here. | |
| Sep 11, 2015 at 10:28 | comment | added | JonB | Well, what about a situation when the sample size is large enough to detect a difference, say an odds ratio of 1.8, but the confidence intervals may be very wide at 1.01-3.9. The power, as formally defined (might) be adequate but the sample size in this case is still too small to get a reliable estimate of the effect of the independent variable of interest. | |
| Sep 11, 2015 at 10:20 | history | answered | Tim | CC BY-SA 3.0 |