Why do we make a big fuss about using Fisher scoring when we fit a GLM? I'm curious about why we treat fitting GLMS as though they were some special optimization problem.  Are they?  It seems to me that they're just maximum likelihood, and that we write down the likelihood and then ... we maximize it!  So why do we use Fisher scoring instead of any of the myriad of optimization schemes that has been developed in the applied math literature? 
 A: Fisher's scoring is just a version of Newton's method that happens to be identified with GLMs, there's nothing particularly special about it, other than the fact that the Fisher's information matrix happens to be rather easy to find for random variables in the exponential family.  It also ties in to a lot of other math-stat material that tends to come up about the same time, and gives a nice geometric intuition about what exactly Fisher information means.
There's absolutely no reason I can think of not to use some other optimizer if you prefer, other than that you might have to code it by hand rather than use a pre-existing package.  I suspect that any strong emphasis on Fisher scoring is a combination of (in order of decreasing weight) pedagogy, ease-of-derivation, historical bias, and "not-invented-here" syndrome.  
A: It's historical, and pragmatic; Nelder and Wedderburn reverse-engineered GLMs, as the set of models where you can find the MLE using Fisher scoring (i.e. Iteratively ReWeighted Least Squares). The algorithm came before the models, at least in the general case.
It's also worth remembering that IWLS was what they had available back in the early 70s, so GLMs were an important class of models to know about. The fact you can maximize GLM likelihoods reliably using Newton-type algorithms (they typically have unique MLEs) also meant that programs like GLIM could be used by those without skills in numerical optimization.
