Timeline for How to do bayesian updating when a statistical test gives insignificant results?
Current License: CC BY-SA 3.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 2, 2017 at 12:02 | comment | added | Hugh | @mrb While the Bayesian can say "A is probably better than B" the frequentist can make statements like "If we repeat the experiment with a sample size of 100 then the sample mean of A will probably be larger than the sample mean of B". This is because sample means are random variables but population means are not (in the frequentist view) | |
| Dec 2, 2017 at 11:59 | comment | added | Hugh | @mrb I think you're trying to apply bayesian ideas to the frequentist style of thinking. In the frequentist view either A is better than B or B is better than A or A and B are equal. There's no validity in saying "A is probably better than B" because the statement "A is better than B" is either true or false: either 100% likely or 0% likely, we cannot talk about probability of truth in the frequentist view of the world. | |
| Dec 1, 2017 at 17:57 | comment | added | mrb | Also, the estimated effect can be huge and still be not statistically significant (it depends on the statistical power of the test). I don't think the effect size matters here. | |
| Dec 1, 2017 at 17:44 | comment | added | mrb | Thanks. I might have said that in the wrong way. But your answer does not fully address my question. After the experiment, should I consider A and B equal, as statistical significance seems to suggest? or A is better than B as Bayesian updating seems to suggest? | |
| Dec 1, 2017 at 17:41 | history | answered | Hugh | CC BY-SA 3.0 |