4
$\begingroup$

I have been asked to calculate a 95% confidence interval based on the following paper: https://www.nber.org/system/files/working_papers/w26107/w26107.pdf

The question reads:

Suppose rainfall in the Bodélé Depression increases by 5 mm. Give a 95% confidence interval for the reduction in PM2.5 concentration in West Africa that you would expect to see in this setting."

The closest relevant piece of information I can find in the paper is on page 11:

Using our historical data, we then estimate that a 1 mm increase in rainfall in the Bodélé during the Harmattan season reduces PM2.5 on average in West Africa by 0.71μg/m³.

The closest I can get to an estimate of this answer is that if every 1mm increase in rainfall causes a 0.71μg/m³ decrease in PM2.5 concentration, then 5 * 0.71 = 3.55μg/m³.

What's not clear to me is how I am meant to find a 95% CI for this. My understanding is that confidence intervals are calculated from the standard deviation and standard error of the original dataset, which I do not have access to. Am I missing something, here?

Edit: Page 24 also mentions:

We estimate that 1 millimeter of additional rainfall in the Bodele reduces PM2.5 in our study locations by an average of 1.2 μg/m3 and therefore our estimate of the coefficient on the rainfall instrument ranges from -1.5 to -2.

This provides an average and a range, but it's still not clear to me how I could work out a confidence interval based on any information in the paper without the original dataset.

$\endgroup$
4
  • $\begingroup$ Please add the self-study tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. $\endgroup$ Commented Jan 24, 2022 at 12:40
  • $\begingroup$ The only relevant information I can find in a quick look through the paper is the plot of nine binned values of (Rainfall, PM2.5) in Figure 3c at p. 20. Each value is given as means and IQRs. The latter show strong heteroscedasticity, which you must account for. You can obtain a rough confidence interval from these data if you are very careful. It will be important to recognize that the entire range of rainfall is only 5mm! Moreover, the line fitted in that plot is about twice as steep as the value of $-1.2$ reported on p. 24. $\endgroup$
    – whuber
    Commented Jan 24, 2022 at 14:59
  • 1
    $\begingroup$ @whuber I've read these values as: 23, 25, 22, 23, 24, 22, 19, 16, 14, 13, 14 PM2.5. This gives a mean of 19.55 and standard deviation of 4.5, for a 95% CI of 16.53 to 22.57. Is this acceptable, or is there a way I should be taking into account the wide IQRs? $\endgroup$
    – jserv
    Commented Jan 24, 2022 at 15:35
  • 2
    $\begingroup$ You need to be doing weighted regression. Your CI doesn't answer your question about how much of a change in PM2.5 is associated with a 5mm change in rainfall. $\endgroup$
    – whuber
    Commented Jan 24, 2022 at 16:27

1 Answer 1

3
$\begingroup$

Figure 3c of the paper appears to have relevant data. I have not read the paper in enough detail to know whether these are the only relevant data, so my intention here is only to reveal the nature of a meaningful answer.

This is my version of the figure.

Figure

The caption reads, in part,

(c) ... Annual observations of dust concentrations over the Bod´el´e were divided into equal sized bins and the average annual PM2.5 concentrations (points) and interquartile ranges (gray lines) over all survey locations were calculated within each of these bins.

The points and gray lines thereby constitute a summary of the data, while the dotted line and the blue region through which it goes are part of my analysis. A dotted line also appears in the original figure and is very close to this one.

I reasoned thus: the interquartile ranges (IQRs) vary substantially across the dataset. This suggests we might be less certain about the averages with large IQRs than about averages with small IQRs. I therefore took the IQR to be a proxy for the standard errors of the averages. The fitted line is the inverse-variance weighted least squares fit, as would be appropriate under this assumption, and the blue polygonal area is a 95% confidence region for the fit. Because it's not an awful approximation to the data, let's continue to interpret the results.

Here is part of the summary of the least squares fit.

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 131.3009    13.6643   9.609 4.98e-06 ***
Rainfall     -2.5802     0.3038  -8.492 1.37e-05 ***

The last line indicates the slope of the response is estimated at $-2.58$ (in micrograms per cubic meter per millimeter of rainfall) with a standard error of $0.3038.$ Consequently, a change in rainfall of $5$ mm would be associated with a change in PM2.5 of $5 \times (-2.58) \approx -12.9$ micrograms per cubic meter with a standard error of $5\times 0.3038 \approx 1.52.$

An approximate two-sided 95% confidence interval can be constructed by putting bounds at a certain multiple $t$ of the standard error around the estimate. Usually, we would use the multiple determined by the Student $t$ distribution with $11 - 2$ degrees of freedom: $11$ for the $11$ data points minus $2$ for the two parameters (intercept, slope) needed to fit the line. This multiple is $2.262,$ giving a confidence interval of $-12.9 \mp 2.262\times 1.52 = [-16.3, -9.46].$

I am suspicious of these weights because an apparent rainfall histogram in the original figure suggests (without any explanation) that very little of the rainfall was observed above 44 mm. That would cause us to suspect the relative precision of the right half of the points might be far less than indicated by their narrow IQRs. To assess this, I repeated the analysis using no weights at all. Although the estimated slope barely changed (from $-2.58$ to $-2.48$), the standard error of its estimate increased to $0.36.$ The resulting confidence interval is $[-16.5, -8.36].$ That is, it is wider at the upper end, with its upper limit increasing from $-9.46$ to $-8.36.$

Figure 2

There aren't enough data points to support fitting a more complex model, even though the data hint that changing the rainfall in the lower area (from $41$ to $43$ mm) might be associated with no change in PM2.5 at all.

The uncertainties associated with binning the data and lack of clear descriptions of what everything in the original plot means all suggest these confidence intervals are too narrow -- but there doesn't seem to be much we can do about that.

Finally, we should be suspicious both of this analysis and of the quoted statements from the paper, because they aren't consistent. The range for the slope in the quotations ($-1.5$ to $-2$) differs from the estimates (around $-2.5$) achievable from Figure 3c and is too narrow for the standard errors of $0.3$ to $0.36$ associated with that figure. Unless you can locate a passage in the paper to explain this discrepancy, use all of its results with care and consider contacting the authors for clarification.


For the record, and for further analysis, here are the numerical values I used (given as an R expression).

X <- data.frame(Rainfall=seq(41, 46, by=0.5),
                PM2_5 = c(24, 25, 21, 24, 24.5, 21, 18, 16, 15, 13.5, 14),
                Q3 = c(32.5, 33.5, 33, 33.25, 33, 30, 23, 17, 16, 14, 18),
                Q1 = c(12, 12, 10, 12, 12, 11.5, 11, 11, 10.5, 10, 11))

The least squares output (for the weighted fit) was obtained with

summary(lm(PM2_5 ~ Rainfall, X, weights = 1/(Q3-Q1)^2))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.