I do not know if you are still interested in this issue (this is an old question). Anyway my attempt to answer might be useful to somebody in the future.
If I read well the original problem, you would like to 'dis-aggregate a coarse histogram'. To make the problem more challenging, the original data come with an 'open group' (> 3000). As pointed in the question, the situation is somehow 'non-standard' (or at least not a text-book example) but this kind of situation is actually really common (in actuarial applications for example).
The way I would approach the problem is by using the penalized composite link model proposed by Eilers (2007). The idea presented in the paper follows, more or less, these lines
- We assume that there is an unobserved (latent) vector of counts say $\gamma$ wich is defined for a finer set of bins
- A matrix $C$ creates the aggregated counts as $\mu = C\gamma$
- What we observe the $m$-dim vector $y$ (i.e. the counts of element 'falling' in a set of bins) are realizaitons of a Poisson process with $E(y) = \mu$ and hence $y \sim \mbox{Pois}(\mu)$
- The matrix $C$ (dim $m \times n$ with $n >> m$) describes how the latent distribution was mixed before generating the data (the Rcode below should clarify how to build it).
In order to estimate the elements of the latent vector of counts $\gamma$ we need make some extra assumption
- The vector $\gamma$ is smooth and $\gamma = \exp(B \beta)$: so we just need to impose that $\beta$, a vector of coefficients, is smooth
- The matrix $B$ is a B-spline matrix built on a set of equally spaced knots (see e.g. Eilers and Marx, 1991)
- The likelihood is $$
\ell = \sum_{i} (y_{i} \log \mu_{i} - \mu_{i}) - \lambda P
$$
- $P$ is a penalty matrix obtained as $P = D^{\top}D$
- $D$ is a finite difference matrix of, say, order 2 (see e.g. Eiles and Marx, 1991)
- $\lambda$ is a regularization parameter to be chosen/selected (in Eilers, 2007 the author uses the AIC criterion to select this parameter if I remember well)
These are the general setting of the solution I would adopt for the original problem. Fortunately, we do not need to do the computations by ourselves. We can use the Rpackage JOPS instead which contains the function pclm (you can download this package here: https://psplines.bitbucket.io/).
Below you will find a code that reproduces your example and plots the inferred latent density for a fixed value of $\lambda$ (you can follow the reference I mentioned before to estimate the optimal value of this parameter). Please also notice that, in the code below, I make the assumption that the class $> 3000$ goes from $3000$ to $4500$. You can change 4500 with the value you prefer (as long as it is a reasonable one for your problem).
library(JOPS)
library(colorout)
library(ggplot2)
# Data
xl = c(60, 100, 200, 300, 500, 1000, 2000, 3000)
xr = c(100, 200, 300, 500, 1000, 2000, 3000, 4500)
y = c(275, 320, 112, 65, 53, 44, 14, 15)
# Set-ups
m = length(y)
n = xr[m]
widths = xr - xl + 1
dens = y/widths/sum(y)
# Composition matrix C 1 if finer bin is included in the
# coarse one, 0 otherwise
C = matrix(0, m, n)
for(i in 1:m) C[i, xl[i]:xr[i]] = 1
C = C[, -c(1:(min(xl)-1))]
# Prepare B-splines bases
x = (n - ncol(C)+1):n
B = JOPS::bbase(x)
# Fit
mod = JOPS::pclm(y, C, B, lambda = 1, pord = 2, show = T)
# Plot
fit = data.frame(x = x, fits = mod$gamma / sum(mod$gamma))
dat = data.frame(xl = xl, xr = xr, y = dens)
plt = ggplot(dat, aes(ymin = 0)) +
geom_rect(aes(xmin = xl, xmax = xr, ymax = y, fill =
I('white'), color = I('black'))) +
geom_line(aes(x = x, y = fits), data = fit, col = I('red'))
+ ylab('Density') + theme_bw()
plt
To conclude, I hope my attempt replies, at least partially, to the original question (and is clear enough ^_^).
