1
$\begingroup$

I observe a bias effect in my measurement system that I can explain and correct using a simple latent or hidden variable model. I am sure this kind of effect has been described earlier in other fields but I cannot find the correct keywords to connect to the published state of the art.

A simplified description (to make things analytically solvabel):

  • We have an unobserved variable $x$ that is drawn from an exponential distribution $$x \sim \pi_\lambda(x) =\lambda\exp(-\lambda x)$$

  • The observed variable $y$, the measurement, conditioned on $x$ follows a normal distribution with mean $x$ $$y \sim \pi_{\sigma}(y|x) = N(y; x, \sigma^2)$$

  • We can use Bayes' theorem to get the distribution of $x$ given measurement $y$ $$\pi_{\lambda, \sigma}(x|y) \propto \pi_{\sigma}(y|x) \pi_\lambda(x) \propto \exp\left(-\frac{1}{2\sigma^2}(y - x)^2 - \lambda x\right) \propto N\left(x; y - \lambda \sigma^2, \sigma^2\right)$$

  • Thus $E[X|y] = y - \lambda \sigma^2$

We can assume $\lambda$ and $\sigma^2$ to be known.In reality, $\lambda$ would be inferred from a larger sample.

$y$ itself is commonly used as the estimator of $x$ in my field. This leads to a biased measurement in situations with a strongly skewed prior distribution (large $\lambda$ in this toy example) and low signal-to-noise ratio ($\sigma$ not $\ll y$).

$y - \lambda \sigma^2$ on the other hand would be an unbiased estimator of the unobserved variable $x$.

My questions are:

  • Is the reasoning layed out in this toy example flawed?
  • Do you know a name for this kind of effect or can you point to a treatment of a similar situation?

I have searched the net for any combination of the terms bias, skewed distribution, estimation, noisy measurement and hidden/latent/unobserved variable or state, but couldn't find a description that really matches this setup.

Thanks for your help!

Edit: Naturally the correct posterior distribution in the above toy example is a truncated normal as $x \geq 0$. Thus $$E[X|y] = y - \lambda \sigma^2 + \frac{\phi\left( \frac{-(y - \lambda \sigma^2)}{\sigma} \right) - 0}{1 - \Phi\left( \frac{-(y - \lambda \sigma^2)}{\sigma} \right)}\cdot \sigma$$ with $\phi$ and $\Phi$ denoting the PDF and CDF of the standard normal, respectively (https://en.wikipedia.org/wiki/Truncated_normal_distribution). Here a plot for $\lambda=\sigma=1$ enter image description here

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.