1
$\begingroup$

I have survey data, with weights that adjust the sample to be nationally representative, from two time points. There is a binary presence/absence variable. Presence increases from time 1 to time 2. I would like to put a confidence interval around the percentage increase. This would be a risk ratio confidence interval, which is straightforward unweighted. But, I can't find any method for a risk-ratio confidence-interval that allows weighting. Is there any known? An R implementation would be ideal.

As a crude method I have thought of calculating confidence intervals around the proportion at time 1 and proportion at time 2, for which weighted methods do exist (https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.4780131009) and then using ratios between these intervals. But (a) I don't know an R implementation, and (b) I don't know how valid this would be be.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Make sure you really want risk ratios. They have multiple drawbacks including (1) a zero-order problem where the ratio varies wildly depending on you take ratios of proportions of presence vs. ratios of proportions of absence and (2) a first-order problem where the ratio must vary as a function of the reference class's risk. On (2) a risk ratio of 2 cannot apply to a base risk > 0.5. You can easily get odds ratios for weighted data using weighted logistic regression depending on how you handle correlations over time. $\endgroup$
    – Frank Harrell
    Commented Jul 14, 2019 at 10:47
  • $\begingroup$ @FrankHarrell Thanks for the point, I'm definitely after risk ratio. They have drawbacks but so do odds ratios, especially for a lay audience. Lay audiences general cope better with statements like "the behaviour was twice as common at time 2". $\endgroup$
    – Amorphia
    Commented Jul 16, 2019 at 10:49
  • 2
    $\begingroup$ I've found over time that it's a mistake to choose a more problematic measure just to please an audience. I always choose the best measure and take time to explain it and to compare with measures they may be more familiar with. $\endgroup$
    – Frank Harrell
    Commented Jul 16, 2019 at 11:24

2 Answers 2

Reset to default
3
$\begingroup$

The survey package for R will let you estimate functions of weighted probabilities.

After you declare a survey design object with your sampling design and weights

means <- svymean(~ outcome_time1 + outcome_time2, design=my_design)
svycontrast(means, quote(outcome_time2/outcome_time1))

There's also a regression function svyglm that allows you to estimate risk ratios (or allows Frank to estimate odds ratios) directly

Improve this answer
$\endgroup$
0
$\begingroup$

I've now done it by bootstrapping, where I used the survey weights as sampling probabilities for by bootstrap samples. I'm not 100% certain this is valid but I have a strong hunch that this is probably OK.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This will be wrong, because you don't actually have more information about observations with higher weight as the bootstrap is assuming, but it typically won't be very wrong. $\endgroup$
    – Thomas Lumley
    Commented Aug 13 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.