In order to compute the power of a statistical test ( eg https://en.wikipedia.org/wiki/Power_of_a_test#Example ) , we need to define "what happens" in the non-null hypothesis. Whereas what happens in the null hypothesis is always simple, what happens in the non-null hypothesis requires to define a priori a value for how different we assume the two populations to be (noted by theta in the link).

How should this theta be computed ? In some tests I've seen in the wild, we take as theta the observed difference of means. Is this OK to do or should the theta be defined before the test ?