6
$\begingroup$

This is from Psychological Science 22(11), p.1363:

  1. If an analysis includes a covariate, authors must report the statistical results of the analysis without the covariate. Reporting covariate-free results makes transparent the extent to which a finding is reliant on the presence of a covariate, puts appropriate pressure on authors to justify the use of the covariate, and encourages reviewers to consider whether including it is warranted. Some findings may be persuasive even if covariates are required for their detection, but one should place greater scrutiny on results that do hinge on covariates despite random assignment.

I don't understand this. Does this mean redoing the statistics after discovering a covariate and excluding that variable (What is the point?)? Or declaring ex ante what is the expected covariate?

Improve this question
$\endgroup$
7
$\begingroup$

At its face this requirement is quite absurd. The most important issue is having a pre-specified, sound, statistical analysis plan. Adjustment variables should usually be pre-specified as part of this plan, and the reasons for adjustment made clear. That makes the covariate-adjusted analysis the "gold standard" and any alternate analysis will just add confusion, and doubt in the minds of reviewers.

Results that hinge on covariates despite randomization are quite common, if too few subjects were randomized to yield sufficient power for an unadjusted analysis. More details can be found in my analysis of covariance chapter in BBR - see http://www.fharrell.com/p/blog-page.html for my links.

Improve this answer
$\endgroup$
  • 2
    $\begingroup$ Very good points. Perhaps you want to be slightly more explicit in answering the OP's questions? $\endgroup$ – Stephan Kolassa Jun 19 '17 at 14:15
  • 1
    $\begingroup$ Please point me to how I didn't $\endgroup$ – Frank Harrell Jun 19 '17 at 16:01
  • 2
    $\begingroup$ You very nicely answered question #2 ("What is the point?"), but not explicitly #1 ("Does this mean redoing the statistics after discovering a covariate and excluding that variable?"), nor #3 ("Or declaring ex ante what is the expected covariate?") $\endgroup$ – Stephan Kolassa Jun 19 '17 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.