# How is this formula for variance derived?

There's a formula for the variance of the traffic flow between A and B, calculated from sample data, quoted in the UK's Traffic Appraisal Manual. No proof is given and part of me really wants to know how this formula is derived. I've spent a good couple of hours trying, but have failed miserably and I can't find anything on the internet either. I would be very grateful for some help.

The form in the Manual is:$$Var(Q_{AB})=\frac{Q(Q-q)}{q^2(q-1)}\cdot{}q_{AB}(q-q_{AB})$$

Where $$Q$$ is the total flow at the survey location, $$q$$ is the flow which is surveyed (i.e. the sample size), $$q_{AB}$$ is the sampled flow travelling from A to B, and $$Q_{AB}$$ is the total flow from A to B.

This can be rearranged into the more intuitive form of:$$Var(Q_{AB})=\frac{(Q-q)\cdot{}Q\cdot{}p_{AB}\cdot{}(1-p_{AB})}{q-1}$$

Where $$p_{AB}$$ is the proportion of sampled traffic travelling from A to B.

Even though this looks more logical, I still can't fathom how this comes from the formula for variance of the binomial distribution, which I thought would be $$Var(Q_{AB})=\frac{p_{AB}(1-p_{AB})}{q-1}$$

Thanks in advance for any help!

For any curious, the section of the manual in question is here:

### Confusion in the notation

There is quite confusion in the formulas. For example, we estimate $$Q_a$$ via $$\frac{q_a}{q}Q$$, and treat it as a random variable. Assuming $$Q,q$$ are known scalars, and $$q_a$$ is a random variable, the source of randomness inside $$Q_a$$ comes from $$q_a$$, and the following holds true:

$$\operatorname{var}(Q_a)=\frac{Q^2}{q^2}\operatorname{var}(q_a)$$

Above formula is a function of $$Q$$ and $$q$$, however, the formula in UK's Traffic Appraisal Manual contains all the terms, i.e. $$Q, q$$ and $$q_a$$. So, the random variable $$q_a$$ appears in the final formula as well. This is like saying $$\operatorname{var}(X)\propto \alpha X$$, which is in general not correct, since the result should've been a scalar instead of a random quantity. If the formula had actually meant $$\operatorname{var}(Q_a|q_a)$$, so that we have the right to use $$q_a$$ in the output expression, then the variance should've been $$0$$, since there is no other randomness left in the expression of $$q_a$$, i.e. $$\operatorname{var}(X|X)=0.$$

So, although the intuition is clear, i.e. figuring out a measure of uncertainty in the estimation of the true number of cars having the attribute of interest, the formula doesn't make sense in the analysis.

### An approximate formula

The situation resembles hypergeometric distribution, although in that $$Q,q,Q_a$$ are known. Here, we don't know $$Q_a$$, and estimate it using a simple formula, which is also the method of moments estimator for $$Q_a$$. Let's say the true number of cars that are of interest is $$R$$ (since we reserve $$Q_a$$ for the estimation, i.e. $$\hat R=Q_a$$). Then, the variance of $$q_a$$ is given in the wikipedia page as follows: $$\operatorname{var}(q_a)=q\frac{R}{Q}\frac{Q-R}{Q}\frac{Q-q}{Q-1}\rightarrow\operatorname{var}(Q_a)=\frac{R(Q-R)(Q-q)}{q(Q-1)}$$

But, we don't know $$R$$, so we can't quantify this formula. A simple thing to do is just plugging in its estimation, i.e. $$R\approx \hat R=Q_a=q_aQ/q$$ in the formula: $$\operatorname{var}(Q_a)\approx \frac{Q}{(Q-1)}\frac{Q(Q-q)}{q^3}q_a(q-q_a)=\frac{Q(q-1)}{(Q-1)q}\underbrace{\frac{Q(Q-q)}{q^2(q-1)}q_a(q-q_a)}_{\text{Given Formula}}$$

This formula is very similar to the one given in the manual, the multiplier in the beginning can actually be assumed close to $$1$$, if both $$q$$ and $$Q$$ are assumed sufficiently large, since $$\frac{q}{q-1}\approx \frac{Q}{Q-1}$$.

### Bayesian way

As I said, this is an approximation to the variance. In order to correctly quantify the variance, a prior distribution over $$R$$ should be present. Then, the formula we found above would be $$\operatorname{var}(Q_a|R)$$, and we could use Law of Total Variance to find the variance of $$Q_a$$:

$$\operatorname{var}(Q_a)=E[\operatorname{var}(Q_a|R)]+\operatorname{var}(E[Q_a|R])$$

Here, the first term is a simple expected value $$E[R(Q-R)]\frac{Q-q}{q(Q-1)}$$, and the second term is $$\operatorname{var}(E[Q_a|R])=\operatorname{var}(E[q_aQ/q|R])=\operatorname{var}(R)$$

both depending on the prior distribution of $$R$$. I don't think there is one explicitly assumed in the document.