Estimating a "true" value with a noisy number of additive noises

Question

I want to recover an estimate of the "true" value of a variable from a small set of noisy observations (5–20). I have an a priori model describing the physical process that generates the observations, but unfortunately it's a complicated one:

$$x_i = 2k_i X + \sum_{j=1}^{2k_i} n_{ij} \qquad i = 1,\ldots,N$$

where $x_i$ is an observation, $X$ is the "true" value to be estimated, $k_i \in 1,2,3,\ldots$ is a discrete noise variable, and $n_{ij} \in [0, +\infty)$ is a continuous noise variable. $k_i$ and $n_{ij}$ are i.i.d., and are independent of each other. Only $\{x_i\}$ are observed. Estimate $X$.

(Physical intuition: A ball is thrown back and forth between two actors $k_i$ times. Each throw adds a random delay, $n_{ij}$. Knowing only the set of total times for the games of catch, $\{x_i\}$, recover the one-way travel time, $X$.)

(The small-sample constraint is for practical reasons: each measurement is fairly expensive.)

I would prefer an answer making as few assumptions about underlying distributions as possible, but we do know a little bit about them. There is a value known in advance, $\kappa$, such that most of the $k_i$ will be equal to $\kappa$, most of the exceptions will be equal to $\kappa+1$, and so on. When $\kappa > 1$, it is theoretically possible for $k_i$ to be less than $\kappa$, but it should almost never happen. The $n_{ij}$ are less well characterized: the best guess I have for their distribution is "some sort of heavy-tailed pseudo-exponential thing," and prior research suggests that they can get really, really messy, with multiple strong modes and so on. The expectation value for $n$ is on the order of $X/10$, but individual values larger than $20X$ will happen. Worse, estimates of the distribution of $n$ done in one locality are worse than useless in another locality. (This is a field study, with global scope.) However, it is sometimes possible to make independent field estimations of $n$ (using another method), whereas $k$ can only be directly observed under lab conditions. (I'm sorry, I can't be more specific about what we are actually measuring.)

I know basic linear regression, and a little bit about mixed models, but the way the number of continuous noises is itself random puts it beyond my knowledge.

Answer 1

You have not defined the problem unambiguously, so I will write down the assumptions with some new assumptions amended (independence and such, come with critique/better assumptions). We have $$ x_i = 2k_i X + \sum_{j=1}^{2k_i} n_{ij} ,\qquad i=1, \dotsc N $$
where $x_i$ is the observed data, $X$ is the unknown quantity to be estimated, $k_1, k_N$ are observed independent random variables with common probability function $p(k), k=1,2,3,\dotsc$, $n_{ij}>0$ are unobserved independent random variables with common density function $f(n)$, also independent of the $k_i$'s.

You want a nonparametric solution, without assumptions on the functions $p(k), f(n)$. One simple solution is the method of moments.

Calculate the expectation of $x_i$: $$ \DeclareMathOperator{\E}{\mathbb{E}} \E x_i = 2 \E k_i \cdot X + 2 \E k_i \cdot \E n $$ then replace the expectations with corresponding arithmetic means, and solve for $X$, which gives as estimator: $$\hat{X} = \frac{\bar{x}}{2\bar{k}} -\bar{n} $$ I will come back to this question and try to find some better solution. One obvious problem with this method of moments estimator is that it can give negative estimated values.