1
$\begingroup$

I am analyzing an engineering problem where the the coefficient of variation (CV) is being used to assess how reliably the quantity of a certain chemical can be measured. The context of this problem is as follows. The engineers have 16 stations and at each station they take about 100 samples. They measure the amount of a specific chemical in the samples. For each of the 16 sites, the sample mean and sdev of the amount of chemical in the samples are computed.

The engineers think there is an inherent problem with the how reliable the measurements are because the CV is very high. I believe the solution to their problem is simply that they are looking at a Poisson process. The basis for this belief is that the chemical measured is produced by scale in a pipe that "flakes off" occasionally (hence the measured quantity can be seen as the sum of a number of discrete events and the conditions for a Poisson process are most likely satisfied).

This hypothesis is testable since for a Poisson distribution the sdev is equal to the square root of the mean. When I make a scatterplot of the sdev vs the sqrt of the mean (for the 16 stations), there is a strong positive linear relation (with a coeff of det about 73% and a p-value for the model about 0.00002)

My problem however is the following: when I look at the estimated slope for the regression model (sdev vs sqrt(Xbar)) it is about 11, although theory predicts it should be unity. I am wondering what would account for this? I do not know of another distribution that has the property that the sdev and the square root of the mean are proportional and I don't think a non-homogeneous Poisson process would produce my observed results.

Addition: when I try to fit the sdev and the mean with a linear model (to see if an exponential model might be appropriate) the fit is also very good (R^2 is about 70% and pvalue for the model is about 0.00005) but the slope again is off but not by as much (is 2.3, but should be unity). Which is at least better and perhaps the discrepancy can be explained by random error (the standard error is about 0.4, so 1 is close to being with a 95% CI for the slope). I'm not sure the model makes sense physically though.

Thanks for any help or suggestions anyone might have,

Matt

$\endgroup$
  • $\begingroup$ Can you describe the data-generating process in more detail? For a "point", what is the $X$? Where do the average and variance come from? $\endgroup$ – GeoMatt22 May 9 '17 at 21:34
  • $\begingroup$ The RV, "X", is the amount of a certain chemical measured. Measurements were taken at 16 locations. At each location, at least 100 samples were collected and the sample mean and sdev from these samples were computed. Therefore, we are considering the sample means and sdevs of the amt of chemical measured at these 16 sites (so we have 16 sample means and sdevs) $\endgroup$ – Matt Brenneman May 9 '17 at 22:02
  • 1
    $\begingroup$ Please edit the question rather than adding clarifying information in comments. It's hard to see why your measurements should be modelled by a Poisson random variable - they don't seem to be discrete counts. $\endgroup$ – Scortchi - Reinstate Monica May 9 '17 at 22:12
  • 1
    $\begingroup$ This could be made to work with any two-parameter distribution family, pretty much. (See also overdispersion.) I just gave the example of log-normal because 1) conveniently, the coefficient of variation is expressed in terms of one of its "natural" parameters, and 2) sometimes a logarithmic scale is natural for chemistry (e.g. pH) $\endgroup$ – GeoMatt22 May 9 '17 at 22:55
  • 2
    $\begingroup$ Poisson processes might be used as a model for the flaking events (so the number of such events could be well approximated by a Poisson) but the amount of material each time would be a random quantity. If you think that the amount at each event is i.i,d from some distribution -- perhaps gamma or lognormal, say -- then the process would be a compound Poisson process & the total amount of the material in some given interval would have a compound Poisson distribution. ... ctd $\endgroup$ – Glen_b May 10 '17 at 3:54
1
$\begingroup$

The Poisson could come into this, but I think some of the ideas in your question are mistaken.

A Poisson process is a counting process -- it might be used as a model for the flaking events (so the number of such events could be well approximated by a Poisson) but the amount of material each time would be a random quantity.

If you think that the amount at each event is i.i.d. from some distribution -- perhaps gamma or lognormal or Pareto, say -- then the process would be a compound Poisson process.

If $X_i$ is the amount released at the $i$th event then the total amount, $Y$ in time $t$ is:

$$Y(t) = \sum_{i=1}^{N(t)} X_i$$

where $N(t)$ is the number of Poisson events in time $t$.

The total amount of the material in some given interval would have a compound Poisson distribution.

If $N\sim \text{Pois}(\lambda)$ then $Y\mid N = X_1+X_2+...+X_N$.

There are other compound models using different count-distributions that the Poisson, of course, but the Poisson is a fairly commonly used one. It has some very simple properties.

Such distributions show up in a number of contexts, and simple facts (like how the mean and variance of the amount in some interval of time relate to the parameter of the Poisson counts and the mean and variance of amounts at each event ("size-distribution") are fairly readily derived via the law of total expectation and the law of total variance.

For the compound Poisson the mean and variance of the unconditional distribution of $Y$ are:

$E(Y) = \lambda\, \mu_X$ where $\lambda$ is the Poisson mean and $\mu_X$ is the mean of the size-distribution.

$\text{Var}(Y) = \lambda\, (\mu_X^2 + \sigma_X^2)$, taking $\sigma_X^2$ as the variance of the size-distribution.

This suggests that the coefficient of variation of a compound Poisson would be

$c_Y = \sqrt{\frac{\mu_X^2 + \sigma_X^2}{\lambda\mu_X^2}} = \sqrt{\frac{1+c_X^2}{\lambda}} = c_N \,\sqrt{1+c_X^2} $

where $c_Y$, $c_X=\sigma_X/\mu_X$ and $c_N=\frac{1}{\sqrt{\lambda}}$ are the coefficients of variation of $Y$, $X$ and $N$ respectively.

This might explain why you would not expect to see slope of 1 in your plot.

$\endgroup$
  • $\begingroup$ Thank you! This is exactly what I was looking for. I knew that it was not a homogeneous Poisson process, but I thought I could sweep this under the rug (and replace the amount by a "mean" amount). I also did not have a good handle on how to deal with the number of events being variable, but now that I look at your soln all my old prob theory is coming back! $\endgroup$ – Matt Brenneman May 10 '17 at 16:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.