11
$\begingroup$

Let us assume we have two parameters, $p_1$ and $p_2$. We also have two maximum likelihood estimators $\hat{p_1}$ and $\hat{p_2}$ and two confidence intervals for these parameters. Is there a way to build a confidence interval for $p_1p_2$?

$\endgroup$
13
$\begingroup$

You can use the Delta method to calculate the standard error of $\hat{p_{1}}\hat{p_{2}}$. The delta method states that an approximation of the variance of a function $g(t)$ is given by: $$ \mathrm{Var}(g(t))\approx \sum_{i=1}^{k}g'_{i}(\theta)^{2}\mathrm{Var}(t_{i})+2\sum_{i>j}g'_{i}(\theta)g'_{j}(\theta)\mathrm{Cov}(t_{i},t_{j}) $$ The approximation of the expectation of $g(t)$ on the other hand is given by: $$ \mathrm{\mathbf{E}}(g(t))\approx g(\theta) $$ So the expectation is simply the function. Your function $g(t)$ is: $g(p_{1}, p_{2})=p_{1}p_{2}$. The expectation of $g(p_{1}, p_{2})=p_{1}p_{2}$ would simply be: $p_{1}p_{2}$. For the variance, we need the partial derivatives of $g(p_{1}, p_{2})$: $$ \begin{align} \frac{\partial}{\partial p_{1}}g(p_{1}p_{2}) & = p_{2} \\ \frac{\partial}{\partial p_{2}}g(p_{1}p_{2}) &= p_{1} \\ \end{align} $$

Using the function for the variance above, we get:

$$ \mathrm{Var}(\hat{p_{1}}\hat{p_{2}})=\hat{p_{2}}^{2}\mathrm{Var}(\hat{p_{1}}) + \hat{p_{1}}^{2}\mathrm{Var}(\hat{p_{2}})+2\cdot \hat{p_{1}}\hat{p_{2}}\mathrm{Cov}(\hat{p_{1}},\hat{p_{2}}) $$ The standard error would then simply be the sqare root of the above expression. Once you've got the standard error, it is straight forward to calculate a 95% confidence interval for $\hat{p_{1}}\hat{p_{2}}$: $\hat{p_{1}}\hat{p_{2}}\pm 1.96\cdot \widehat{\mathrm{SE}}(\hat{p_{1}}\hat{p_{2}})$

To caluclate the standard error of $\hat{p_{1}}\hat{p_{2}}$, you need the variance of $\hat{p_{1}}$ and $\hat{p_{2}}$ which you usually can get by the variance-covariance matrix $\Sigma$ which would be a 2x2-matrix in your case because you have two estimates. The diagonal elements in the variance-covariance matrix are the variances of $\hat{p_{1}}$ and $\hat{p_{2}}$ while the off-diagonal elements are the covariance of $\hat{p_{1}}$ and $\hat{p_{2}}$ (the matrix is symmetric). As @gung mentions in the comments, the variance-covariance matrix can be extracted by most statistical softwares. Sometimes, estimation algorithms provide the Hessian matrix (I won't go into details about that here), and the variance-covariance matrix can be estimated by the inverse of the negative Hessian (but only if you maximized the log-likelihood!; see this post). Again, consult the documentation of your statistical software and/or the web on how to extract the Hessian and on how to calculate the inverse of a matrix.

Alternatively, you can get the variances of $\hat{p_{1}}$ and $\hat{p_{2}}$ from the confidence intervals in the following way (this is valid for a 95%-CI): $\mathrm{SE}(\hat{p_{1}})=(\text{upper limit} - \text{lower limit})/3.92$. For an $100(1-\alpha)\%$-CI, the estimated standard error is: $\mathrm{SE}(\hat{p_{1}})=(\text{upper limit} - \text{lower limit})/(2\cdot z_{1-\alpha/2})$, where $z_{1-\alpha/2}$ is the $(1-\alpha/2)$ quantile of the standard normal distribution (for $\alpha=0.05$, $z_{0.975}\approx 1.96$). Then, $\mathrm{Var}(\hat{p_{1}}) = \mathrm{SE}(\hat{p_{1}})^{2}$. The same is true for the variance of $\hat{p_{2}}$. We need to covariance of $\hat{p_{1}}$ and $\hat{p_{2}}$ too (see paragraph above). If $\hat{p_{1}}$ and $\hat{p_{2}}$ are independent, the covariance is zero and we can drop the term.

This paper might provide additional information.

$\endgroup$
  • 4
    $\begingroup$ +1. The variances of the parameters & their covariance can be found by examining the variance-covariance matrix of $\boldsymbol\beta$, which most statistical software can provide. Eg, in R, it's ?vcov; & in SAS, covb is added as an option to the model statement in PROC REG. $\endgroup$ – gung - Reinstate Monica Jun 29 '13 at 18:11
  • 1
    $\begingroup$ @gung On a point of pedantry it might be worth pointing out (because I know it confuses some people) that it's really the variance-covariance matrix of $\hat \beta$ rather than $\beta$ (and in fact it isn't even really that, because the standard deviation has to be estimated from the sample, so it's really the estimated variance-covariance matrix..) $\endgroup$ – Silverfish Dec 21 '15 at 16:32
  • 2
    $\begingroup$ @Silverfish, duly chastised. Next time I'll say "the estimated variance-covariance matrix of $\boldsymbol{\hat\beta}$". $\endgroup$ – gung - Reinstate Monica Dec 21 '15 at 17:54
  • 1
    $\begingroup$ You could try to construct a profile likelihood function! and construct the confidence interval from that. $\endgroup$ – kjetil b halvorsen Sep 30 '17 at 13:21
  • $\begingroup$ Isn’t $var(p_1)=0$ since it’s a parameter? $\endgroup$ – user0 Dec 7 '18 at 16:48
1
$\begingroup$

I found a different equation for calculation of variance of product.

If x and y are independently distributed, the variance of the product is relatively straightforward: V(x*y)= V(y)*E(x)^2 + V(x)*E(y)^2 + V(x)*V(y) These results also generalize to cases involving three or more variables (Goodman 1960). Source: Regulating Pesticides (1980), appendix F

Coolserdash: The last component V(x)*V(y) is missing in your equation. Is the referenced book (Regulating Pesticides) wrong?

Also, both equations might not be perfect. "... we show that the distribution of the product of three independent normal variables is not normal." (source). I would expect some positive skew even in the product of two normally distributed variables.

$\endgroup$
0
$\begingroup$
  1. The length of the CI / 2 / 1.96 = se, i.e. the standard error of A or B
  2. se^2 = var, i.e. the variance of the estimate A or B
  3. Use the estimated A or B as the means of A or B, i.e. E(A) or E(B)
  4. Follow this page http://falkenblog.blogspot.se/2008/07/formula-for-varxy.html to get var(A*B), i.e. var(C)
  5. Square-root of var(C) is the se of C
  6. (C - 1.96*se(C), C + 1.96*se(C)) is the 95% CI of C

Note that if your A and B are correlated, you need to consider their covariance as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.