What is the appropriate statistical test for pairwise comparisons across different points in time?

Question

I'm wondering what the best statistical test for the following situation:

I have a table that looks like the following:

| Subject | Prior Datetime  | Prior Value | Post Datetime   | Post Value |
+---------+-----------------+-------------+-----------------+------------+
| 1       | 4/2/2020 15:16  | 36.6        | 4/3/2020 2:04   | 69.7       |
| 2       | 4/2/2020 8:27   | 25.8        | 4/3/2020 4:23   | 64.6       |
| 3       | 3/29/2020 3:41  | 10.4        | 3/30/2020 3:02  | 93.0       |
| 4       | 3/25/2020 11:45 | 28.2        | 3/25/2020 14:44 | 96.6       |
| 5       | 3/26/2020 0:39  | 49.9        | 3/26/2020 6:30  | 66.1       |
| 6       | 3/25/2020 14:18 | 31.8        | 3/26/2020 4:38  | 83.1       |
| 7       | 3/26/2020 10:17 | 11.2        | 3/27/2020 5:50  | 83.8       |
| 8       | 3/21/2020 13:39 | 20.2        | 3/21/2020 21:27 | 83.8       |
| 9       | 3/22/2020 15:48 | 25.7        | 3/23/2020 10:15 | 76.7       |
| 10      | 3/21/2020 14:27 | 8.6         | 3/21/2020 23:11 | 64.6       |
| 11      | 3/24/2020 1:32  | 41.3        | 3/24/2020 7:33  | 75.4       |
| 12      | 3/22/2020 12:47 | 32.6        | 3/22/2020 20:15 | 72.9       |
| .       | .               | .           | .               | .          |
| .       | .               | .           | .               | .          |
| .       | .               | .           | .               | .          |
+---------+-----------------+-------------+-----------------+------------+

Basically, the subjects received treatment X between the Prior Datetime and the Post Datetime (and this administration timestamp is available), and the values at those times were taken, although the specific times at which they were done are not consistent (it could have been 5 minutes before or after treatment X, or 25 hours before or after treatment X).

My hypothesis is that the treatment X increases the value of the test in question - which is what we are seeing in the data empirically. What is the best statistical test to test for this?

I've had a few considerations

• It must be pairwise for obvious reasons
• I can't use something like a paired t-test/Wilcoxon test because there is a 'variable' element of time instead of just a Pre/Post treatment (or would this be the best way of doing it?).
• Is it best to deal with timestamps, or should I convert to an 'offset' variable (i.e. number of seconds different from the treatment)?
• Could I treat this as a modeling/regression problem, where the time offset is the independent variable, and the value is dependent? How do you incorporate the paired nature of the data?

I can extract a control group from the data with a similar timeframe for subjects that didn’t receive the treatment.

Answer 1

So you have pre-post data and want to study the effects of some treatment. Assuming you also have a control group, and maybe some other covariables, you can start with an ancova model of the form $$\tag{*}\label{*} y_{\text{post}} =\mu+\alpha y_{\text{pre}}+\gamma \text{Treatment}+\beta x+\text{Error} $$An alternative is change scores defined by $y_{\text{post}}-y_{\text{pre}}$. To look at that subtract $y_{\text{pre}}$ from both sides in $\eqref{*}$: $$ y_{\text{post}}-y_{\text{pre}} =\mu+(\alpha-1) y_{\text{pre}}+\gamma \text{Treatment}+\beta x+\text{Error} =\mu+\alpha y_{\text{pre}}+\gamma \text{Treatment}+\beta x+\text{Error} $$ But, when using change scores $y_{\text{pre}}$ is typically not included on the RHS, so that use is equivalent to assuming that $\alpha=1$ in $\eqref{*}$. That way using change scores is a special case of ancova. In the model $\eqref{*}$ we could also include, say, an interaction between treatment and the pre score, which is not possible when using change scores. There is an earlier post Best practice when analysing pre-post treatment-control designs with good answers and many references, and ancova seems to be the recommendation, at least for randomized studies. For non-randomized studies the situation is less clear, see this paper.

But there is an additional complication, that you have observational data and not experimental data. There could be systematic differences between the treatment and control group. Hopefully you have many additional covariates, and if those covariates can explain pre-differences between the two groups, maybe there is no problem. But you could look into propensity scores, many posts, search this site. And maybe add the tag causality which might attract the attention here from knowledgeable users on that topic. Register-based data analysis is an example, see this.