I've been struggling with this for a while now, hopefully someone will know how to help me :)
Here it is :
1) I'm using a linear mixed effects model on longitudinal data (biological values of many patients over time). Basically, the model predicts a quatitative outcome from the treatment group (2 groups) and the time since treatment initiation.
2) Note that i force the model to deny any baseline difference between groups, by removing the treatment effect by itself => the treatment effect is thus only present as an interaction with time. Apparently this is legit in my case, because there is no reason for the treatment to impact baseline value since it has "just started".
3) Anyways, now i want to predict an "average" trajectory for each treatment group : i thus simulate the treatment by "fixing" the treatment group for every point in time, and i obtain a prediction for each group at every time mark. If i plot this, i obtain to curves (lines) with similar intercept but different slopes.
4) Now i've only performed the simulation once for each treatment at each point in time. If i repeat the process 1000 times, I obtain a distribution of predictions for each group at every timemark. I extract the 2.5 and 97.5 percentiles of my predictions, which gives me a 95% prediction interval, for each group at every point in time.
5) I have one particular time mark that I'm interested in : i thus have 2 prediction means with prediction intervals (one for each treatment group).
Here's the question : how do I compare those 2 prediction intervals ? Regular test use the standard error, which is linked to the number of observations, but in my case observations are just simulations. What i mean is I could easily Perform 1 million simulations and thus obtain a somewhat "artificial" statistical difference.
I'm pretty sure I got something wrong somewhere, just not sure where ...
Thanks in advance for your help !