4
$\begingroup$

The appendix of the paper of McPherson et al (1982) (see screenshot below) contains a derivation of the Systematic Component of Variation (SCV). I understand the derivation with exception of the first step. Here are the premises:

$O_i$: observed cases in region i
$E_i$: expected cases in region i
$\lambda_i$: multiplicative factor associated with region i ($O_i=\lambda_i*E_i$)

Now the following assumptions have been made:

$O_i$ is approximately Poisson distributed with mean $\lambda_iE_i$
$\lambda_i$ is considered as a random variable with expected value $1$ and variance $\sigma^2$.

From these the following formula is concluded:

var($O_i$) = $E_i^2\sigma^2$ + $E_i$

I tried to find out how to get the formula by the given premises and assumptions and didn't succeed. Any idea?

Screenshot of McPherson's derivation:

enter image description here

$\endgroup$
7
  • $\begingroup$ I'm a bit surprised that this simple question has languished 8 months without an answer. $\endgroup$ Dec 16, 2016 at 16:09
  • $\begingroup$ It looks more like 33 months to me, wasn't it asked in April 2014 or am I misreading it? I only came across it by chance as I'm getting my (decidedly non-mathematical) brain around the technique for work. $\endgroup$
    – Tumbledown
    Dec 16, 2016 at 16:19
  • 1
    $\begingroup$ @kjetil and Tumbledown: I put the question in October 2016. There is something wrong in the time setting of SO. $\endgroup$
    – giordano
    Dec 17, 2016 at 15:10
  • 2
    $\begingroup$ @Tumbledown "Apr 14" means the 14th day of April in the current year. If an earlier year than the current one is intended, it is shown using a format like: Jan 26 '12 (26th day of January, 2012) $\endgroup$
    – Glen_b
    Dec 17, 2016 at 22:14
  • $\begingroup$ I was convinced I aksed the question in October 2016. Maybe I was wrong. @Glen_b Why not be consequently using mmm dd yyyy? (My favorite would be yyyy-mm-dd or dd.mm.yyyy). $\endgroup$
    – giordano
    Dec 18, 2016 at 14:46

1 Answer 1

4
$\begingroup$

This is simply the law of total variance: https://en.wikipedia.org/wiki/Law_of_total_variance

You have given:

$E_i$ is a known constant

$\DeclareMathOperator{\E}{\mathbb{E}}\E \lambda_i = 1$

$\DeclareMathOperator{\V}{\mathbb{V}}\V \lambda_i=\sigma^2 $

$\E(O_i | \lambda_i)=\lambda_i E_i$

$\V(O_i | \lambda_i) = \lambda_i E_i$

Using this with the law of total variance we get: $$ \V (O_i) = \E \V (O_i | \lambda_i) + \V \E (O_i | \lambda_i) \\ = \E (\lambda_i E_i) + \V (\lambda_i E_i) \\ = E_i \E(\lambda_i) + E_i^2 \V (\lambda_i) \\ = E_i + E_i^2 \sigma^2 \\ = E_i (1+\sigma^2 E_i) $$

$\endgroup$
4
  • $\begingroup$ I'm very happy that this question got answered. Thank you very much. Indeed, the mathematical arrangement is simple. I found the statistical assumptions difficult. The important point for me in your answer was: $E_i$ is a known constant. But isn't $\mathbb E(O_i) = \lambda_i E_i $? $\endgroup$
    – giordano
    Dec 17, 2016 at 15:26
  • 1
    $\begingroup$ The question in comment : NO, $\E (O_i | \lambda_i)=\lambda_i E_i$, but $\E O_i = E_i$, since $\E \lambda_i=1$. $\endgroup$ Dec 17, 2016 at 15:30
  • 1
    $\begingroup$ You saved my day. $\endgroup$
    – giordano
    Dec 19, 2016 at 7:41
  • $\begingroup$ @kjetilbhalvorsen They should have been really more verbose. The equation after "so that" is also strange given that the left hand side is a (non-degenerate) random variable, while the right hand side is a deterministic value, and also the next line is a simple rearrangement, yet $\sigma$ is miraculously replaced with $\widehat{\sigma}$. $\endgroup$ May 26, 2020 at 13:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.