5
$\begingroup$

Let's say we have the following setup:

There are $n$ participants doing a simple task: They watch a video. At random times, a symbol is shown somewhere on the screen. The total number of times the symbol is displayed, say $N$, is fixed and the same for all participants. At the end, the participants have to provide their guesses of the total number of times they saw the symbol during the video. Let's call this variable $N_{obs,i}$ with $i=1,2,\ldots, n$. We assume that each instance of the symbol has the same detection probability. Also, the information whether or not a single symbol was detected is not available, just the total from each participant.

The detection probabilities differ between participants. So some subjects might count the symbols corrects while other might under- or overestimate the total number of symbols.

Example

Let's say $N=29$. 20 subjects participate in the study. The following $N_{obs}$ are recorded: $29, 28, 29, 29, 27, 30, 28, 26, 28, 28, 27, 29, 28, 28, 29, 26, 29, 29, 28, 25.$

Question

How could a Bayesian model for estimation of the true total $N$ from the estimated numbers $N_{obs,i}$ look like? This question assumes that the researcher that analyzes the data doesn't know the true $N$, of course.

Were I'm stuck

Applying Bayes' Theorem, I know that:

$$ P(N|N_{obs}) = P(N_{obs}|N)\times \frac{P(N)}{P(N_{obs})} $$

I have trouble fleshing out the details. $P(N)$ is the prior for the total number of symbols (a uniform distribution, for example). The likelihood $P(N_{obs}|N)$ is binomial, but I have trouble putting all the pieces together.


Edit

Thanks to @Kodiologist's great answer, I was able to implement his model in JAGS. I first randomly generated 20 observations (see above under "Example") according to the model. Then, I fitted the model.

Here is the model in JAGS:

model{
    for(i in 1:N) {    
      obs[i] ~ dsum(a[i], b[i])

      a[i] ~ dbin(theta, Nt)
      b[i] ~ dpois(lambda[i])
      lambda[i] ~ dt(0,1,1)T(0,) # Half-Cauchy prior   
    }    
    theta ~ dbeta(5, 1) # Informative prior leaning towards 1
    Nt ~ dcat(p[])        
    }

Here are the simulated data:

# Simulated data
set.seed(123)
win.data <- list(
  obs = rbinom(20, 29, 0.95) + rpois(20, 0.5)
  , N = 20
  , p = rep(1/60, 60)
)

Some initial values and the fit using JAGS:

inits <- function(){
  list(
    lambda = abs(rt(20, 1)),
    # alpha = rgamma(1, 1, 1),
    # beta = rgamma(1, 1, 1),
    theta = rbeta(1, 1, 1),
    b = rpois(20, 1),
    a = rbinom(20, 29, 0.5),
    Nt = sample(29:50, 1)
  )
}    

params <- c("lambda", "theta", "Nt")

# MCMC settings
nc <- 5         # Number of chains
ni <- 20000     # Number of draws from posterior per chain
nb <- 10000     # Number of draws to discard as burn-in
nt <- 2         # Thinning rate

out <- jags(
  data = win.data,
  parameters.to.save = params,
  model.file = "pandas.txt",
  n.chains = nc,
  n.iter = ni,
  n.burnin = nb,
  n.thin=nt,
  inits=inits,
  progress.bar="text")

Here is the relevant output (cropped):

           mu.vect sd.vect   2.5%    25%    50%    75%  97.5%  Rhat n.eff
Nt          28.807   0.939 28.000 28.000 29.000 29.000 31.000 1.017   260
theta        0.943   0.030  0.870  0.930  0.948  0.965  0.981 1.016   310

So it seems it worked really well.

$\endgroup$
  • $\begingroup$ If I'm reading this correctly, you want to use only one subject's observed count, and so n is irrelevant. Is that right? $\endgroup$ – Kodiologist Oct 7 '16 at 21:26
  • $\begingroup$ What specifically are you asking for here? An explicit formula for the posterior? A sense of what the posterior will look like compared to the prior? $\endgroup$ – Accidental Statistician Oct 7 '16 at 23:58
  • $\begingroup$ @Kodiologist No, there are multiple subjects. Ideally, the model incorporates all subjects' guesses. $\endgroup$ – COOLSerdash Oct 8 '16 at 6:09
  • $\begingroup$ @AccidentalStatistician Yes, the ideal would be an explicit posterior or an idea how one could get that specifically. $\endgroup$ – COOLSerdash Oct 8 '16 at 6:10
  • $\begingroup$ @COOLSerdash Okay. One other question: you said that each instance of the symbol has the same detection probability, but is this constant between subjects, or might some subjects be more observant than others? $\endgroup$ – Kodiologist Oct 8 '16 at 6:32
3
$\begingroup$

This seems like a hard problem, especially considering how subjects might overestimate the number of symbols, which I'd previously thought you assumed not to happen. You may need some fairly informative priors in order to get precise posterior estimates. (Or you can add assumptions to simplify the model.)

Here's one idea for a model. Let $k_i$ be the number of symbols that subject $i$ reports. Suppose $k_i = a_i + b_i$, where $a_i$ is the number of real symbols the subject noticed, and $b_i$ is the number of additional symbols the subject wrongly thought he saw. Each $a_i$ comes from a binomial distribution with $N$ trials and probability of success $θ_i$, which itself comes from a beta distribution with parameters $α$ and $β$. Each $b_i$ comes from a Poisson distribution with parameter $λ$. (For yet more complexity, you can replace the universal $λ$ with a per-subject parameter $λ_i$.) So you need priors for $N$, $α$, $β$, and $λ$, and you're primarily interested in the posterior distribution of $N$.

I doubt it's possible to fit this model analytically. Use an MCMC sampler such as Stan instead.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.