3
$\begingroup$

I'm reading the book STAT2: Modeling with Regression and ANOVA. In Section 5.6, the authors consider what to do when you want to do a one-way ANOVA but the group standard deviations are not equal. They introduce a technique for deciding which power $p$ to use in a transformation $y \mapsto y^p$ whose goal is to transform so that group standard deviations are close enough to equal to do ANOVA. Here is the technique, which the authors credit to Tukey:

  1. Compute group means $\overline{y_i}$ and group standard deviations $s_i$.
  2. Plot $\log(s_i)$ versus $\log(\overline{y_i})$, one point per group.
  3. Fit a line to this scatterplot. If the line fits well, there is a re-expression that will make the group standard deviations more nearly equal. (If not, not.)
  4. Let $slope$ be the slope of the line above, let $p = 1-\ $slope, and do the transformation $y\mapsto y^p$ (but, if $p = 0$ do $\log(y)$).

Can someone explain a theoretical reason why $p = 1-\ $slope solves the problem?

I'd be happy to understand this even in the case of just two groups, so that the regression line always fits perfectly. It is clear that, if the two group standard deviations are already equal, then the slope above will be 0 and the procedure will correctly pick $p = 1$. For two groups where things differ by an order of magnitude, I have some heuristic argument involving the relationship between orders of magnitude of the $s_i$ and orders of magnitude of the $\overline{y_i}$ but I am not happy with it. I did some simulations where I'd expect specific values of $p$ (e.g., start with two groups with equal standard deviations, square everything, then do the procedure above), but the procedure was often giving me values of $p$ that were slightly off, even in the absence of random noise. Can someone please explain why this procedure works, in theory? Thanks!

P.S. Please feel free to retag. And, if anyone knows which Tukey paper introduced this procedure, that would help too. Googling "1 minus slope" didn't yield much.

$\endgroup$

1 Answer 1

2
$\begingroup$

The equation found with the log-log plot

$$\log(s_i) = a + b \log(\bar{y}_i)$$

is equivalent to

$$s_i = e^{a} \cdot \bar{y}_i^b$$

So the standard deviation follows a relationship that is a power of the observations $y_i$.


If we apply the transformation $y^{1-b}$ then this is approximately

$$y^{1-b} \approx \bar{y}_i^{-b} + (1-b) \bar{y}_i^{-b} (y-\bar{y}_i)$$

With this linear approximation the difference $(y - \bar{y}_i)$ is scaled by the term $(1-b) \bar{y}^{-b} $ so the new deviation will be approximately

$$s_{i,new} \approx (1-b) \bar{y}_i^{-b} s_{i,old}$$

and in the relationship as function of $\bar{y}_i$

$$s_{i,new} \approx (1-b) \bar{y}_i^{-b} (e^{a} \cdot \bar{y}_i^b) = (1-b)e^{a} $$

$\endgroup$
3
  • $\begingroup$ Thanks for the answer! Can you please say more about the first approximations you used? Why is $y^{1-b} \approx \overline{y_i}^{-b} + (1-b)\overline{y_i}^{-b} (y-\overline{y}_i)$? Do you mean that this is true for all groups $i$? For the second approximation, I guess this is because $Var[cX] = c^2 Var[X]$ so $s_{new} = c s_{old}$. Everything else is clear to me. Thanks! $\endgroup$ Oct 11, 2021 at 11:17
  • 2
    $\begingroup$ @DavidWhite it is the linear approximation of the power law function. It is based on the first two terms of a Taylor series expansion $$f(y) = f(\hat y_i) + f'(\hat y_i) (y-\hat y_i) + \frac{1}{2} f''(\hat y_i) (y-\hat y_i)^2 + \frac{1}{6} f'''(\hat y_i) (y-\hat y_i)^3 + ...$$ with $f(y) = y^{1-b}$ you get the result. See a very related use of the approximation here en.wikipedia.org/wiki/Delta_method $\endgroup$ Oct 11, 2021 at 12:12
  • $\begingroup$ Thanks! That is perfect. $\endgroup$ Oct 11, 2021 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.