I have a bunch of count data associated with time intervals (potentially overlapping and of variable lengths), say
$(s_i, t_i, n_i, N_i)$
where $N_i$ is a count of the total number of events observed by a particular observer over a time interval $[s_i, t_i]$, and $n_i$ is the number of those events of some class of interest.
I'm interested in modelling how the proportion of interesting events varies over time, assuming that this proportion is independent of the observer. The total number of events observed in each interval may vary somewhat arbitrarily based on the observer, although if necessary we can safely assume that in aggregate over all observers it gives a reasonable picture of the overall event density.
The simplest model I could think of for this data would be to assume $n_i \sim Binomial\left[N_i, \frac{1}{t_i-s_i}\int_{s_i}^{t_i}\theta(w) dw\right]$
For some unknown smooth function $\theta(t)$ which lies in $[0,1]$ for all $t$, and look for non-parametric methods to estimate $\theta$.
A slightly better model could perhaps be to assume an inhomogeneous Poisson process for the overall events with rate $\lambda(t)$, and that the events of interest are a thinning of this process in a time-varying proportion $\theta(t)$ yielding another inhomog. Poisson process with rate $\lambda(t)\theta(t)$. We can further assume that both these processes are thinned further by each observer, independently to the "event of interest" thinning, in some unknown fixed proportion representing the proportion of events of both types which the observer sees. This proportion cancels out leaving:
$n_i \sim Binomial\left[N_i, \frac{\int_{s_i}^{t_i}\theta(w)\lambda(w) dw}{\int_{s_i}^{t_i} \lambda(w) dw}\right]$.
Here $\lambda(t)$ only matters up to a constant factor, so perhaps it would be better to model it as a density rather than a Poisson process rate? Anyway the simpler model falls out as a special case of this if you assume $\lambda(t)$ is constant/uniform.
I was wondering if anyone has any pointers on how to estimate the proportion function $\theta(t)$ in this kind of model?
A naive ad-hoc guess would be to compute a separate estimate for the overall rate and the rate of interest using kernel smoothing methods, then compute a ratio of these two estimates (perhaps adding on some small constants akin to what you'd get using a Beta prior to estimate a standalone binomial proportion e.g. $\frac{n+\alpha}{N+\alpha+\beta}$).
To bring out the heavy guns, I suppose one could put a logistic Gaussian Process prior on $\theta$ and estimate look for a posterior estimate. I imagine this would require some fairly hairy approximate Bayesian methods.
I was wondering there were any simpler frequentist estimators for this (based for example on kernel density estimators), and if anything is known about hyper-parameter selection, convergence rates, confidence intervals and so on.
If it helps to consider a simpler problem first, we could perhaps assume that we have point observations for the events (of interest and not).
Thanks!
