Partial correlation coefficient inhabits the domain of linear relationships/regression. You admitted this yourself when giving the definition for partial r in your question. Partial r is just another way of standardazing the linear regression coefficient, the other way being the standardized coefficient beta. So, partial r cannot exist in the context other than that where usual (zero-order) Pearson r exists; it itself is the Pearson correlation, only refined after washing out some "irrelevant" information from it by means of linear algebra.
Spearman rho - as you might be aware - is just Pearson r computed on ranked data rather than raw data. So, as long as you agree to treat the ranks as the "raw data" (that is, treat ranking as just preprocessing) you may carry the concept of partial r, including interpretation, over to Spearman rho.
Situation with Kendall tau is different. This coefficient, unlike Spearman's, is not based on linear correlation/regression. It has its own ideology maths and interpretation, these of Goodman-Kruskal gamma. Therefore, notion of partial r is inapplicable to it, and if you apply that recursive formula you mention to a matrix of tau's that will mean that you believe its entries are Pearson r's!. If there is possible a proper analogue of "partial correlation" for tau, it must be computed by a very different formula exploiting the concept of conditional probability of co-occurrence instead of linear regression between residuals.