Experiment design: Researchers ran an experiment to test whether paying people to get up on time made them less likely to sleep in late. Fifty participants were randomized to 2 groups: a control group who got no incentive, or a group who got a small daily payment. The main outcome measure was the average time participants got out of bed after 16 weeks. Results: Compared with the control group, the daily payments group got up a mean of 9.2 minutes earlier (t = 1.21; 95% CI, -3.20 to 12.66; P = .23) than in the no-incentives group (4.4 minutes).
I would recommend not paying, of course. I would also recommend changing the way that the results are reported. "There was insufficient evidence to conclude that an incentive of X was successful at reducing the average time participants got out of bed."
Lastly, if the researcher wants to continue down this line of research, there are a couple of ways to change the experiment:
- Create an experimental design with varying incentive. All that was really proved is that the particular incentive wasn't successful. Trying larger incentives and analyzing the results with a regression might suggest the incentive necessary for a statistically significant result.
- Try a different type of incentive all together. e.g. cash instead of coupons.
- Try a different end point. If wake-up time is too variable, maybe try bed time.
- I don't recommend this, but some researchers will just try the experiment again with a larger sample size under the assumption that the difference really was there, but they didn't have enough sample. I think it is a much better path to success to vary the incentive than just to increase the sample size to find again that there was no significant effect.