Spearman rank correlation is just Pearson correlation applied to ranks, a point often obscured by the emphasis on the simple computational short-cut formula for Spearman that is found in many books. So, I wouldn't rule out Fisher's z procedures for Spearman. There is a caution that the sampling distribution will differ at least a bit with Spearman -- as the sampling distribution of Spearman is irregular in detail -- and indeed that could bite hard with small sample sizes. But most things are problematic with small sample sizes. The caution that everything hinges on the data being treated as ranks is already the caution that applies to Spearman correlation.
I've got to suggest, however, that this line of enquiry may not prove very fruitful for you.
For a start, it is usually better to bring two groups into the same model if you can. Also, one rank correlation being stronger than another leaves the more important question of what the relationships and differences are, quantitatively, a bit in the background.
The data have to be really irregular for there to be no transformation or link function (logarithm? square root? reciprocal?) that won't bring them into reasonable shape for some kind of ANOVA or more general(ized) linear model. As it seems you have some kind of experiment, that would probably mesh better with your scientific objectives too.
(LATER) You did say "ordinal" and that is important. Much depends on what that means precisely. If ordinal means a five-point scale, something like an ordered logit or probit model may be appropriate. If ordinal means judgment-based scores of some kind, much depends on how they behave.
