The canonical link function is not chosen because it's logical (or "successful" whatever that means in this case) for any particular purpose (though in many cases it works out that way) -- it relates to a specific feature of the way the mean parameter appears in the density function.
If a link function doesn't suit what you do, you shouldn't use it.
In particular, if $g(\mu|\mathbf{x})$ is not linear in $\mathbf{x}$ ... you have the wrong link.
The reciprocal link function in exponential regression doesn't constrain to positive values,
It does within the range of the data but can go outside it if you extrapolate.
If it's important to avoid, you might consider whether a log-link is a better choice.
(a) Regarding the lack of constraint of results to non-negative predictions, my thinking is that, as long as we choose an initial trial parameter vector that results in a non-negative linear predictor, the chances of ending up with negative predictions is negligible since the sheer number of points in the neighbourhood of zero will make the predictions very accurate in that region.
Yes, in practice, it usually isn't a problem.
(b) I am not clear as to how transforming values close to zero to be spread out over the range of the reciprocal function makes the regression more successful. I can see that it reduces leverage compared with an ordinary regression to fit the means, but is there something deeper connected with it being easier to fit a linear function to points that are reasonably far apart?
keen to get some more understanding around what makes a good link function over and above that it constrains predictions to appropriate values.
Well, usually the main criterion is that the linear predictor is linear; if you have only factors for predictors that's obviously not a direct concern (I'd think more about things like interactions -- a good choice of link may make the model additive in the main effects which has some advantages).
However, getting feasible predictions in some part of the range may be important. In some cases this indicates a problem with the model (you may in fact know that the $E(Y|\mathbf{x}^{(0)})=0$ at some special value ofr the set of predictors, $\mathbf{x}^{(0)}$ for example, which suggests you should incorporate that knowledge in the model. In other cases you want it to produce only positive predictions for any possible set of $x$'s in which case you might look at different links (or indeed to some other way of imposing the constraint).
