The details are mostly explained a couple of pages earlier, at the start of sec 5.8, and that in turn refers back to chapter 3 (in particular, see sec 3.9, especially 3.9.2).
Consider $\text{Var}(y)=\phi V(\mu)$ and the transformation $y^*=h(\mu)$. What effect does that have on the variance, approximately?
Under suitable conditions, we can obtain an approximation of the moments of transformed random variables via Taylor expansion.
Using a one-term approximation as given at that Wikipedia link:
$$\text{Var}(y^*)=\text{Var}\!\left(h(y)\right)\approx \left(h'(\mu _{y})\right)^{2}\sigma _{y}^{2}=\left(h'(\mu _{y})\right)^{2}.\phi V(\mu _{y})$$
How do we stabilize variance? That is, how do we choose $h$ to make $\text{Var}(y^*)$ approximately constant? By making $\left(h'(\mu _{y})\right)^{2}\propto V(\mu _{y})^{-1}$ $\ldots\:{(1)}$, so that the two functions of $\mu$ in the expression $\left(h'(\mu _{y})\right)^{2}\cdot\phi\cdot V(\mu _{y})$ cancel, so that this approximation to the variance is then a constant function of $\mu$.
So now back to Sec 5.8. What GLM specification for $y$ approximately corresponds (in terms of mean and variance) to taking the transformation $y^*=h(y)$ and then fitting a constant-variance linear regression?
If $E(y^*)=E[h(y)]$ is linear in $\eta = β_0 + β_1x_1 + ··· + β_px_p=X\beta$ (e.g. see sec 1.6 between eqn 1.5 and 1.6 and the first equation in 4.6.1), then the mean as a function of the $x$'s needs to be such that $g(\mu)=β_0 + β_1x_1 + ··· + β_px_p$ (see sec 1.6 p 13)
But now consider just the first term in the Taylor expansion about the mean;
$E[h(y)] \approx h(\mu)$ (see the section on the first moment in the Wikipedia link); so we will need that the link function $g(\mu)\approx h(\mu)$.
In the transformed regression we also had $\text{Var}(y^*)=c$, so if $h$ was a variance-stabilizing transformation, then $\text{Var}(y)\propto 1/h'(μ)^2$, just as in eqn $(1)$ above.
