Can rank-ordering of response variables be used with generalized linear models? From a certain point of view, a standard linear regression finds a model which minimizes the Pearson correlation between the predicted value of the response variable and the true value of the response variable. Are there any (standard) modifications to the procedure which instead minimize a rank-order correlation (e.g. the Spearman rho or Kendall tau)? Or more generally, are there approaches to generalized linear models where the functional form of the link function is unknown/unspecified?
I have a situation where I think a generalized linear model is likely to be appropriate ... except that I don't know what the functional form for the link function would be. I can, however, specify the constraint that the link function will be monotonic. Hence the hope that I can fit a (generalized) linear model to rank ordering (which will just enforce monotonicity) rather than a standard fit (which will also attempt to match absolute magnitudes of the response variable).
An approach which can be regularized (as in lasso or ridge) would be much preferred.
Note: My situation is distinct from that for ordinal regression. My response variable is a continuously-valued real function, rather than a small set of discrete integer levels as is the case for ordinal regression.
 A: After thinking about it, I realized ordinal regression is indeed appropriate for this situation.
While the response variable is a continuous real, it isn't an infinitely precise continuous real. In all likelihood, it's coming from some physical measure that has an associated uncertainty. Even if the measurement were to be infinitely precise, the formulation of the regression problem implies that there is some measurement inaccuracy in the response variable, making the precision a false one. Finally, if you're only concerned about rank order, any change in precision is "pointless" as long as it doesn't change the effective order of the entries.
For example, imagine you're modeling the weight of widgets in a manufacturing process. Theoretically, there weight is a continuous real, but the scale on which you measure them has a fixed precision (e.g. milligrams). There's also a practical level of intrinsic variability (e.g. two widgets within a tenth of a gram of each other might be considered "the same weight" with respect to the downstream process.). Finally, there's the limitations of the input data. If one widget in the training set is 5.57 g and then next heavier is 5.89 g, no rank-order based method will show a practical distinction between a prediction of 5.65 g versus 5.75 g. 
In this light, it should be possible to choose some "precision" under which changes in the response variable do not substantially affect the point of the regression problem. You can then "round to the nearest" to get discreet response levels you can then subject to ordinal regression.
The one consideration may be the number of output levels with respect to the number of input variables. Apparently, good ordinal regression packages are able to handle situations where the number of output levels is on the same order or even equal to the number of input cases. (That is, if each input case has it's own unique output level.) As I understand it, the total ordering of the output levels is a rather weak requirement, in terms of information costs, so you don't really risk over fitting with a large number of response levels.
With discretized output levels - especially with one level per input case - ordinal regression becomes practically equivalent to generalized linear regression with an arbitrary monotonic link function.
