1
$\begingroup$

I am collecting one-year impacts to conduct a meta-analysis on a certain type of education intervention (high-dosage tutoring). Some papers only report two-year impacts of the intervention and the corresponding standard errors. Is it possible for me to obtain an estimate of the annual impact and its standard error from just the reported two-year impact and its standard error? We are assuming that the distribution of the outcome variable remains the same over the two years of the intervention.

An example: a paper only offers the two-year cumulative impact (let's say $\hat{\beta} = 0.6$ with $SE(\hat{\beta}) = 0.4$) but I want the annual impact. Would an appropriate estimation of the one-year impact be $\hat{\beta/2} = 0.3$ with $SE(\hat{\beta}/2) = 0.2$?

$\endgroup$
  • 1
    $\begingroup$ Yes, you are right. $\endgroup$ – Patrick Li Nov 24 '15 at 15:32
  • $\begingroup$ I guess it feels weird because you can then divide impacts from an experiment into arbitrarily small time-intervals (one-week, one-day, etc.) and get tiny standard errors. It doesn't feel like the estimation for a one-day impact should be that precise. $\endgroup$ – Tim Nov 24 '15 at 15:37
  • 1
    $\begingroup$ That's an interesting distinction, Tim. There can be a strong difference between half of a two-year estimate and an estimate for one year! If your objective is to provide a one-year impact estimate, then perhaps you should reformulate your question to ask about that--and when you do, please provide information about the data, procedures, models, and assumptions you (or the paper's authors) are making in order to compute these estimates. $\endgroup$ – whuber Nov 24 '15 at 16:08
  • $\begingroup$ @whuber Thanks. I have updated the question to better reflect my specific issue. $\endgroup$ – Tim Nov 24 '15 at 16:35
1
$\begingroup$

If

${{SE}(\hat{\beta}) = \sqrt{{VAR}(\hat{\beta})}}$

Then

${SE}(\hat{\beta/c}) = \sqrt{{VAR}(\hat{\beta}/c)}= \sqrt{{VAR}(\hat{\beta})/c^2} = \sqrt{{VAR}(\hat{\beta})}/c$

EDIT:

This part is now obsolete since you've changed your question. For your new question, the answer depends very much on your outcome, and how you assume the "intervention" to affect it.

Dividing by 2 is suitable when you assume the effects of tutoring are additive. I.e. receiving a second year of tutoring has the same absolute effect as receiving the first year.

If you assume the effects to be multiplicative (e.g. each year of tutoring increases the outcome-measure by 10%) then you should choose another transformation of $\hat{\beta}_{2yr}$.

In summary: the ideal choice depends very much on the details of your measure, your assumptions about what the intervention does and most importantly the regression model you're estimating.

$\endgroup$
  • $\begingroup$ Thanks for your answer. I have changed the question to better reflect my specific issue. As whuber mentioned above, I believe there might be a difference between dividing an estimate by a constant and obtaining an estimate for one-year impact. $\endgroup$ – Tim Nov 24 '15 at 16:53
0
$\begingroup$

Your estimate for the mean 1-year beta was correct, however, standard errors are proportional to the square root of the time step. I would therefore expect the 1 year estimate of the standard error to be 0.4 / sqrt(2).

This intuitively makes sense, too, as you would expect greater uncertainty around results in a shorter timeframe.

$\endgroup$
  • $\begingroup$ Thanks. Do you have a source for this? $\endgroup$ – Tim Nov 24 '15 at 17:06
  • $\begingroup$ This answer appears to assume the results over the two successive years are uncorrelated--but that is scarcely plausible. $\endgroup$ – whuber Nov 24 '15 at 18:00
  • $\begingroup$ @whuber So is this correct way to think about it then... if we assumed that they are uncorrelated the divisor is $\sqrt(2)$. If we assume they are perfectly correlated, the divisor is 2. In the real world, both of these assumptions are probably not correct and the actual divisor would fall between those numbers? $\endgroup$ – Tim Nov 24 '15 at 19:56
  • $\begingroup$ I worry that it could be worse than that. There are plausible circumstances in which the divisor could be close to zero and others in which is would be nearly infinite. To make any progress you need some kind of model of how the impact develops over time, as well as of the possible errors in its measurement. Consider (by analogy) a study of weight loss among dieters. Conceivably after two years they all had returned to their original weights, even though they had lost considerable weight the first year. You can't estimate the one-year response just from the two-year response! $\endgroup$ – whuber Nov 24 '15 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.