2
$\begingroup$

I'd like to assess the effect of a medication X on some scalar, nominal and ordinal parameters of disease Y.

Problem is, this medication X has proven to greatly reduce mortality in patients with disease Y, so depriving patients from this drug is a no-go. There is no other drug for this disease.

The way our data collection system is set up, I have data on those parameters at baseline, that is when patients are diagnosed with disease Y and start taking drug X. I can also measure these parameters at any point while they take this medication. I'm currently thinking of running a paired t-test for the scalar and ordinal variables and chi-squared tests for the nominal variables.

So, I'd like to ask; are there more powerful statistical tests for these hypotheses? is there a better way to set up such an experiment?

I'm really grateful for your insights in advance!

$\endgroup$
1
  • $\begingroup$ I tried to give a more general overview in my answer because I think they are still many possibilities to consider, before dwelling to the "which test to use". I would be happy to clarify it further if there are any bits and bobs you think they are more applicable to your case. (Welcome to CV!) $\endgroup$
    – usεr11852
    Feb 8, 2019 at 22:52

1 Answer 1

2
$\begingroup$

I think the task described is more general than just simply saying: "Use test A instead of test B". I tried to contextualise things in my answer because I think zooming onto the "which test to use" question is a bit misleading at this point.

The situation where it is impossible to have a control sample is consider a common case of counter-factual inference, i.e. a "what-if" scenario. There is rather large bibliography on the matter that mostly stems from Epidemiology as in many cases it was associated with policy interventions (e.g. what is the effect of a country-wide vaccination programme) where a control sample was impossible. I have used the book by Morgan & Winship "Counterfactual and Causal Inference" extensively as a first reference and I have found it clear and easy to follow. There are also nice open-access articles on this matter like Höfler (2005) "Causal inference based on counterfactuals" or Arhen et al. (2009) "Estimating the effects of potential public health interventions on population disease burden: a step-by-step illustration of causal inference methods" offering general background.

Particular to the use-case described, it is reasonable to assume that we have a longitudinal study design and try to exploit this. I would strongly suggest looking into a regression analysis approach so we can account for time-varying and/or subject specific effects. If anything we would expect some within-subject correlation of measurements as well as some time-dependency between successive measurements. Using a generalised least squares (R-function gls) or a linear mixed model (R-function lmer) would be an obvious choice to control such dependency structures. This will enable us to control for a rump-up period, different treatment strategies, different variable baselines time-varying explanatory variables, etc. in a straightforward way. In addition, we should look to bootstrap our data to get any statistical significance estimates. I cannot be more specific because I do not know the exact research questions but this should set the general tone.

While mentioned that "everyone gets medication X" I suspect that not everyone got it in the past or in other locations. If these data are available it might be possible to have an artificial control study (where maybe some other patients took medication Z). I have recently read Hernan and Robins (2016) "Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available" which touches on a multitude of points like: the definition of time zero, eligibility criteria, follow-up periods, etc. that one needs to be aware of when using "big data" to emulate an infeasible RCT.

Finally, if we have data on the patient before the diagnosis we might also want to consider an interrupted time-series approach, a particular type of a regression discontinuity design. In that case, the working assumption would be that the variable of interest would continue its current trend and the change of the trend is due to the intervention. Again the concept of ramp-up period or time to initiate treatment can be rather informative. Bernal et al. (2017) "Interrupted time series regression for the evaluation of public health interventions: a tutorial" is a very easy to follow if you want to consider this avenue further.

$\endgroup$
2
  • $\begingroup$ That's a very well thought out reply, it'll take a bit for me to make sense of it fully! The interrupted time series approach sounds particularly useful, as I do have some data on patients prior to their diagnosis! $\endgroup$ Feb 10, 2019 at 7:35
  • $\begingroup$ I am glad I could help! If you find this answer helpful you could consider upvoting it or if it answers your question, accept it as an answer. If you need further clarifications you are welcome to ask. $\endgroup$
    – usεr11852
    Feb 12, 2019 at 10:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.