Lets's have at look at both options.
diff(log(x)) calculates relative changes. This also takes care of exponential trends. For example, you would use this to detrend the stock price development of Google. According to the logarithms laws:
$$log(a) - log(b) = log(a/b)$$
all.equal(log(3) - log(5), log(3/5))
This means, instead of using the absolute difference for detrending you are using the relative change. As a bonus differences calculated using the natural logarithm can also be interpreted as a precentage change. For more information I recommend:
Cole, T. J., & Altman, D. G. (2017). Statistics Notes: Percentage differences, symmetry, and natural logarithms. BMJ, 358(August), j3683. https://doi.org/10.1136/bmj.j3683
On the other hand
log(diff(x)) calculates the absolute differences before the logarithm is applied. If you calculate a trend using this method, the trend would be more outlier resistant (but this also applies to
diff(log(x))). This is helpful if there are a small number of big jumps in the time-series. Beware this method would potentially break your analyses when the difference is 0 or negative. (in R:
log(0) = -Inf or
log(-1) = NaN)
In my opinion
diff(log(x)) is the better default choice. While there probably is a use-case for
log(diff(x)), it's quite hard to think of one.