There are different kinds of ordinal variables. Your variable has an underlying ratio scaled variable age and therefore holds more quantitative information than ordinal variables where no such underlying variable is known. With use in Gower's distance my first impulse would be to use this as a quantitative variable where you score each age category by its midpoint. If you have information about the within-class distributions of age you can do better, for example use the mean or compute the midpoint for the highest age class using the maximum of 80 if you know nobody older than 80 is in the sample (just as an example for use of information if you know such a thing).
The standard approach for ordinal data in Gower would be to score the categories 1,2,3,4 as you write, and indeed this loses some information which is retained with the scheme above. The computation of Gower's distance, although advertised as suitable for ordinal data, in fact doesn't do any reasoning that is specifically ordinal, it does a quantitative computation with scores, and these scores are chosen 1,2,3,4 as default in absence of any better information. If you have better information, use it (see above).
A proviso is that in general I recommend that the distance you use should reflect the meaning of the data relative to the clustering aim. Using midpoints of age ranges would score you categories as 6, 15.5, 22, 58. This implies that the effective distance between the 26-90 and 19-25 categories is much larger than the effective distances between the younger categories. If in your situation you have reasons to think that most relevant development happens at young age and there is really not that much going on anymore at higher age that is relevant to your problem, you may prefer coding as 1,2,3,4 because this makes the effective distance between any two neighbouring classes the same, which may well reflect how age actually works in your situation. Regarding the resulting clustering, this will have the effect that there is less tendency to isolate the highest age group and separate it from the others. This may well be appropriate but depends on whether in terms of what you are interested in age should rather count in a quantitative way, or whether its effect flattens out with age and therefore you would not want to treat the larger ages as "quantitatively" remote from 19-25, say.
If you start to think more about this, based on even more background information that you might have, you may well end up thinking that you'd want a scoring that is different from both scorings mentioned up to now, i.e., neither 6, 15.5, 22, 58 nor 1,2,3,4. This is fair enough if you have the expertise and good arguments, however the two scorings mentioned above have the advantage that they are "easy to establish and defend", so to say, whereas if you chose an even different one, you'd need to work harder to convince your audience that and why your specific choice makes sense.
Regarding the R-code in the question, by the way, my arguments make reference to the meaning of the data, and simulated data have no meaning. One can simulate data from the same ordinal distribution using a different (from normal) underlying latent distribution and different intervals, even of same length if you want, so as long as there is no relevant meaning of the variables is involved, it is not mathematically identifiable what the "true" underlying intervals are and whether they are of same length or not.
I should also say that my comments specifically apply to Gower's distance and any approach which treats ordinal variables as quantitative variables with specific scores, in which case you can manipulate the scores in this way, but there are methods that treat ordinal variables differently.
