Why does Tukey's "1 - slope rule" work for transformations in ANOVA?

Question

I'm reading the book STAT2: Modeling with Regression and ANOVA. In Section 5.6, the authors consider what to do when you want to do a one-way ANOVA but the group standard deviations are not equal. They introduce a technique for deciding which power $p$ to use in a transformation $y \mapsto y^p$ whose goal is to transform so that group standard deviations are close enough to equal to do ANOVA. Here is the technique, which the authors credit to Tukey:

1. Compute group means $\overline{y_i}$ and group standard deviations $s_i$.
2. Plot $\log(s_i)$ versus $\log(\overline{y_i})$, one point per group.
3. Fit a line to this scatterplot. If the line fits well, there is a re-expression that will make the group standard deviations more nearly equal. (If not, not.)
4. Let $slope$ be the slope of the line above, let $p = 1-\ $slope, and do the transformation $y\mapsto y^p$ (but, if $p = 0$ do $\log(y)$).

Can someone explain a theoretical reason why $p = 1-\ $slope solves the problem?

I'd be happy to understand this even in the case of just two groups, so that the regression line always fits perfectly. It is clear that, if the two group standard deviations are already equal, then the slope above will be 0 and the procedure will correctly pick $p = 1$. For two groups where things differ by an order of magnitude, I have some heuristic argument involving the relationship between orders of magnitude of the $s_i$ and orders of magnitude of the $\overline{y_i}$ but I am not happy with it. I did some simulations where I'd expect specific values of $p$ (e.g., start with two groups with equal standard deviations, square everything, then do the procedure above), but the procedure was often giving me values of $p$ that were slightly off, even in the absence of random noise. Can someone please explain why this procedure works, in theory? Thanks!

P.S. Please feel free to retag. And, if anyone knows which Tukey paper introduced this procedure, that would help too. Googling "1 minus slope" didn't yield much.

Answer 1

The equation found with the log-log plot

$$\log(s_i) = a + b \log(\bar{y}_i)$$

is equivalent to

$$s_i = e^{a} \cdot \bar{y}_i^b$$

So the standard deviation follows a relationship that is a power of the observations $y_i$.

---

If we apply the transformation $y^{1-b}$ then this is approximately

$$y^{1-b} \approx \bar{y}_i^{-b} + (1-b) \bar{y}_i^{-b} (y-\bar{y}_i)$$

With this linear approximation the difference $(y - \bar{y}_i)$ is scaled by the term $(1-b) \bar{y}^{-b} $ so the new deviation will be approximately

$$s_{i,new} \approx (1-b) \bar{y}_i^{-b} s_{i,old}$$

and in the relationship as function of $\bar{y}_i$

$$s_{i,new} \approx (1-b) \bar{y}_i^{-b} (e^{a} \cdot \bar{y}_i^b) = (1-b)e^{a} $$