7
$\begingroup$

Problem definition

I have a dataset composed by $m$ observations $y^{(1)},\dots,y^{(m)} \in \mathbb{R}^2$ generated as follow \begin{equation*}\begin{aligned} y &= z + v \newline z & \sim\mathcal{U}([a,b]) \newline v & \sim\mathcal{N}(0_{2\times1}, \sigma_v^2 I_2) \end{aligned} \end{equation*} here $z$ is uniformely distributed over the line connecting the vertices $[\begin{array}{cc}a & 0 \end{array}]$, $[\begin{array}{cc}b & 0\end{array}]$ with $b>a$, while $v$ is a zero-mean Gaussian variable with isotropic covariance $\sigma_v^2 I_2$ ($I_2$ is the $2\times 2$ identity matrix). The two components $z$ and $v$ are independent.

Given the dataset $y^{(1)},\dots, y^{(m)}$, I would like to estimate the parameters $a$ and $b$ that define the segment generating $z$.

My attempt

For $\sigma_v\triangleq 0$, i.e. no Gaussian noise, if I'm not wrong the maximum likelihood estimates of $a$ and $b$ should be \begin{equation}\begin{aligned} \hat{\alpha} &\triangleq \min_{j=1,\dots,m} [\begin{array}{cc} 1 & 0 \end{array}] y^{(j)} \newline \hat{\beta} &\triangleq \max_{j=1,\dots,m} [\begin{array}{cc} 1 & 0 \end{array}] y^{(j)} \end{aligned} \tag{$\star$} \end{equation} Now, if $\sigma_v$ is "small" compared to the segment length $b-a$, then the estimates above should work pretty well. As $\sigma_v$ becomes comparable with $b-a$, I expect a degradation of the estimate accuracy. I'm thinking about a sort of heuristic correction as follows \begin{equation*}\begin{aligned} \hat{\alpha} &\triangleq \min_{j=1,\dots,m} [\begin{array}{cc} 1 & 0 \end{array}] y^{(j)} +\sigma_v\newline \hat{\beta} &\triangleq \max_{j=1,\dots,m} [\begin{array}{cc} 1 & 0 \end{array}] y^{(j)} -\sigma_v \end{aligned} \end{equation*} because due to the Gaussian noise $v$ the minimum and the maximum can overshoot $a$ and $b$.

Question

My solution is far from rigorous and I'm not even sure if it makes sense. My questions are the following:

  1. Based on some solid principle, is it possible to correct the estimates $(\star)$ in order to take into account the effect of $v$?
  2. If my approach cannot work, how do we estimate $a$, $b$?
$\endgroup$
3
  • $\begingroup$ Do you know $\sigma_v$ or are you estimating it from the data? $\endgroup$
    – Henry
    Commented Mar 11 at 12:18
  • $\begingroup$ I know $\sigma_v$ $\endgroup$
    – matteogost
    Commented Mar 11 at 13:39
  • 2
    $\begingroup$ In that case you can ignore the second dimension. $\endgroup$
    – Henry
    Commented Mar 11 at 14:16

1 Answer 1

9
$\begingroup$

When you add Gaussian noise, the smallest and largest observations are no longer sufficient for and MLEs of $a,b$.

Instead consider the likelihood which, in the usual way, is given by the product of the density of each $\mathbf{y}_i$, in this case given by \begin{align} f(\mathbf{y})&=\int_a^b f(\mathbf{y}|z)f(z)dz \\&=\frac1{2\pi\sigma_v^2}\int_a^b e^{-\frac1{2\sigma_v^2}[(y_1-z)^2+y_2^2]}\frac1{b-a}dz \\&=\frac1{\sqrt{2\pi}\sigma_u}e^{-\frac{y_2^2}{2\sigma_u^2}}\frac{\Phi(\frac{y_1-a}{\sigma_u})-\Phi(\frac{y_1-b}{\sigma_u})}{b-a}, \end{align} where $\Phi$ is the standard normal cdf.

The resulting likelihood (or its log) should be straightforward to maximise numerically with respect to $a, b, \sigma_u^2$.

I would expect the MLEs of $a$ and $b$ to be biased, however, just like in the case of uniform observations without Gaussian noise, so some form of bias correction may be needed.

An alternative is Bayesian inference, perhaps using an improper uniform prior $\pi(\theta_1)\propto 1$ on $\theta_1=\frac{a+b}2$ and an independent scale prior $\pi(\sigma_u)\propto \frac1{\sigma_u}$ on $\sigma_u$. It would be tempting to use an independent, improper scale prior also on $\theta_2=b-a$ but I believe this would make the posterior improper (as in Hobert (1996)) as the likelihood tends to a positive limit as $\theta_2 \rightarrow 0$. A sensible way around this would be to assume that $\sigma_u$ and $\theta_2$ instead are of similar orders of magnitude, e.g. by letting $\pi(\theta_2|\sigma_u)$ be lognormal with a suitable variance.

The argument for the conditional informative prior $\pi(\theta_2|\sigma_u)$ also applies if $\sigma_u$ is known. Thus, completely "objective" Bayesian inference does not seem feasible for this model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.