A principled way to do this type of comparison is via mixed effects regression modelling.
In your example, you have subjects measured at multiple time points on the covariate of interest (xy). So you can fit a mixed effects regression model where xy is the response variable, Gender and time (t) are predictor variables and ID is a random grouping factor.
What type of mixed effects model you can fit will depend on the nature of your covariate of interest. For example, if that covariate is expressed on a (0,1) scale, a mixed effects beta regression model will likely be appropriate.
You will need to decide whether your model should include an interaction between Gender and Time, a random intercept for subject (as well as a random slope for time?), etc.
Once you fit your chosen model, you can pick a few relevant time points for each gender and "predict" the value of your response variable for that gender and for those time points *for the typical subject" (i.e., a subject for whom the random effect(s) associated with the random grouping factor ID in your model is(are) set to 0). The "prediction" would need to be made on the "response" scale (i.e., the (0,1) scale).
Mixed effects models such as beta regression have a "conditional" interpretation which refers to a "typical" subject. If you wanted your interpretation to be "marginal" (so that it refers to what happens on average across all subjects represented by the ones included in your data), you would have to do extra work.
