6
$\begingroup$

I am currently working with data from camera-trapping, where individuals from multiple species have been taken pictures of when they pass in front of one of the cameras (which are placed on a large area, about 2km apart from each other)

I expect the results (number of pictures taken, segregating by species and habitat type) to follow a Poisson distribution with an unknown parameter. My plan is to estimate it with sample means and then perform tests that check the equality between different habitats. I have tried non-parametric tests, but the samples are small and they don't have enough power

I am not sure, though, about the assumption of a Poisson distribution (it makes sense, but that doesn't mean it's true) When checking for variance among cameras in similar conditions, it consistently turns out to be slightly higher than the mean, which I guess is bad news. What could be a good way to test whether the data fits a poisson distribution or not? Also, what are the exact benefits, in terms of hypothesis test power, of "getting the distribution right" with regards to using non-parametric tests?

I know those are two separate questions, but they come from the same problem and I don't see any point in duplicating the above explanation in another post.

CLARIFYING NOTE: What I get is a dataset where each row indicates the date/time where a photo was taken, the species appearing on it, the camera that took the picture and the type of habitat where it is placed. Then I aggregate the data by both camera, species and day (thus aggregating the effect of day-night cylces), so that each camera-species-day count is a realization of a Poisson distribution (that depends only of habitat and species)

$\endgroup$
  • 4
    $\begingroup$ Could you describe your data more clearly? Are you, for instance, collecting counts of photos by day or by some regular period? Or are you perhaps aggregating counts among all the cameras? Providing a rationale for your Poisson assumption would also help us understand your question, because on the face of it the appearance of a species before a camera is unlikely to be a homogeneous Poisson process: wouldn't most animals exhibit different rates of activity at different times of day? $\endgroup$ – whuber Jun 3 at 13:27
  • 1
    $\begingroup$ +1 Thank you for the extra information; this is an interesting question. $\endgroup$ – whuber Jun 3 at 15:12
6
$\begingroup$

It is almost never a good idea to use a Poisson distribution (or any other single parameter distribution) for modelling --- this distribution fixes the variance in relation to the mean, which means that the model does not allow the estimated variability to conform to the data. For this type of data, you should use a negative binomial model instead. This is a generalisation of the Poisson distribution (coming from mixtures of the Poisson) that has an additional parameter affecting the variance.

It is also worth noting that, in cases like this, some analysts use "over-dispersion tests", but even these are unnecessary unless there is a strong a priori theoretical reason to prefer a Poisson model. There is no strong a priori reason to prefer the Poisson in this case, so I would recommend starting with a negative binomial model. This model will estimate the variance from the data, and so the estimated variability will conform to the data.

$\endgroup$
  • 1
    $\begingroup$ Indeed, there is a strong theoretical reason to model this as a Poisson process, since that would be the distribution we'd have if the animals' poistions were random $\endgroup$ – David Jun 4 at 10:13
  • 1
    $\begingroup$ (+1) I'd note, though, that studies are often contrived to eliminate - well, reduce to negligible levels - heterogeneity: its presence hints at factors that can be controlled or measured & modelled. And - I think this is what @David is saying in the comment above - the whole point might be to test the adequacy of the model. $\endgroup$ – Scortchi - Reinstate Monica Jun 4 at 11:00
  • 1
    $\begingroup$ @David: The animals positions are almost certainly neither uniform, nor independent of one another. Hence, I doubt there is any good theoretical reason to believe in a Poisson distribution. $\endgroup$ – Reinstate Monica Jun 4 at 12:14
  • $\begingroup$ @Ben But the data comes from per-day aggregation on a similar time of the year, so I see no reason to believe that there are "special days" $\endgroup$ – David Jun 4 at 12:53
  • 3
    $\begingroup$ Deviation from the Poisson is likely to come from two sources: (1) the fact that different animals appear with different rates, so you are aggregating over different rates; and (2) the fact that different days have different seasons/temps, etc., and so also give different rates. It is therefore likely that the distribution will be some kind of mixture of Poisson distributions, such as the negative binomial. $\endgroup$ – Reinstate Monica Jun 4 at 22:45
0
$\begingroup$

Consider $n$ counts having the same covariate pattern (replicates). Suppose the $i$th count $X_i$ were to follow a Poisson distribution with its own mean $\mu_i$; the joint mass function of $n$ independent counts is then

$$\begin{align} f_{\vec{X}}(\vec{x};\mu_1,\ldots,\mu_n) &= \prod_{i=1}^n\frac{\mu_i^{x_i}\exp(-\mu_i)}{x_i!}\\ &= \prod_{i=1}^n \frac{1}{x_i!}\cdot\prod_{i=1}^n \mu_i^{x_i}\cdot \exp\left(-\sum_{i=1}^n\mu_i\right) \end{align}$$

Reparametrize with $\phi=\sum_{i=1}^n \mu_i$ & $\pi_i=\frac{\mu_i}{\sum_{i=1}^n \mu_i}$, & let $t=\sum_{i=1}^n x_i$:

$$\begin{align} f_{\vec{X},T}(\vec{x},t;\pi_1,\ldots,\pi_n,\phi)&=\prod_{i=1}^n\frac{1}{x_i!}\cdot\prod_{i=1}^n\pi_i^{x_i}\cdot\phi^{t}\exp(-\phi) \end{align}$$

The statistic $t$ is sufficient for $\phi$ (& ancillary for $\pi_1,\ldots,\pi_n$), with a marginal Poisson distribution having the mass function

$$f_T(t;\phi)= \frac{\phi^t\exp(-\phi)}{t!}$$

So the conditional distribution of $X$ given $T=t$ is multinomial, free of the nuisance parameter $\phi$, having the mass function

$$ f_{X|T}(x;t,\pi_1,\ldots,\pi_n)=\frac{f_{\vec{X},T}(\vec{x},t;\pi_,\ldots,\pi_n,\phi)}{f_{T}(t;\phi)}={\prod_{i=1}^n \frac{t!}{x_i!}}\cdot\prod_{i=1}^n\pi_i^{x_i} $$

With the null hypothesis $\pi_1=\ldots=\pi_n=\frac{1}{n}$, the log likelihood ratio test statistic is

$$2\sum_{i=1}^n X_i \log \frac{X_i}{t/n}$$

& Rao's score test statistic is

$$\sum_{i=1}^n \frac{(X_i-t/n)^2}{t/n}$$

Under the null, both follow, asymptotically, a chi-squared distribution with $n-1$ degrees of freedom ; the resulting tests of goodness of fit are known as the G-test (I don't know why) & Pearson's chi-squared test respectively. Exact distributions of either statistic under the null can be of course be obtained by exhaustive calculation or simulation.

The extension to an omnibus test across $m$ covariate patterns is straightforward, & gives as test statistics

$$2\sum_{j=1}^m\sum_{i=1}^{n_j} X_{ij} \log \frac{X_{ij}}{t_j / n_j}$$

&

$$\sum_{j=1}^m \sum_{i=1}^{n_j} \frac{(X_{ij}-t_j/n_j)^2}{t_j /n_j}$$

both, under the null, asymptotically, following a chi-squared distribution with $\sum_{j=1}^m (n_j-1)$ degrees of freedom.

The score test is, I think, the most popular—the ratio of the sample variance to the sample mean has an intuitive appeal as a measure of extra-Poisson dispersion. Low values may be taken as evidence for under-dispersion.

A variance of counts exceeding their mean may also indicate zero inflation. See Van den Broeck (1995), Biometrics, 51, pp 738–743, "A Score Test for Zero Inflation in a Poisson Distribution" & How to test for Zero-Inflation in a dataset?.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.