I observe a bias effect in my measurement system that I can explain and correct using a simple latent or hidden variable model. I am sure this kind of effect has been described earlier in other fields but I cannot find the correct keywords to connect to the published state of the art.

A simplified description (to make things analytically solvabel):

  • We have an unobserved variable $x$ that is drawn from an exponential distribution $$x \sim \pi_\lambda(x) =\lambda\exp(-\lambda x)$$

  • The observed variable $y$, the measurement, conditioned on $x$ follows a normal distribution with mean $x$ $$y \sim \pi_{\sigma}(y|x) = N(y; x, \sigma^2)$$

  • We can use Bayes' theorem to get the distribution of $x$ given measurement $y$ $$\pi_{\lambda, \sigma}(x|y) \propto \pi_{\sigma}(y|x) \pi_\lambda(x) \propto \exp\left(-\frac{1}{2\sigma^2}(y - x)^2 - \lambda x\right) \propto N\left(x; y - \lambda \sigma^2, \sigma^2\right)$$

  • Thus $E[X|y] = y - \lambda \sigma^2$

We can assume $\lambda$ and $\sigma^2$ to be known.In reality, $\lambda$ would be inferred from a larger sample.

$y$ itself is commonly used as the estimator of $x$ in my field. This leads to a biased measurement in situations with a strongly skewed prior distribution (large $\lambda$ in this toy example) and low signal-to-noise ratio ($\sigma$ not $\ll y$).

$y - \lambda \sigma^2$ on the other hand would be an unbiased estimator of the unobserved variable $x$.

My questions are:

  • Is the reasoning layed out in this toy example flawed?
  • Do you know a name for this kind of effect or can you point to a treatment of a similar situation?

I have searched the net for any combination of the terms bias, skewed distribution, estimation, noisy measurement and hidden/latent/unobserved variable or state, but couldn't find a description that really matches this setup.

Thanks for your help!

Edit: Naturally the correct posterior distribution in the above toy example is a truncated normal as $x \geq 0$. Thus $$E[X|y] = y - \lambda \sigma^2 + \frac{\phi\left( \frac{-(y - \lambda \sigma^2)}{\sigma} \right) - 0}{1 - \Phi\left( \frac{-(y - \lambda \sigma^2)}{\sigma} \right)}\cdot \sigma$$ with $\phi$ and $\Phi$ denoting the PDF and CDF of the standard normal, respectively (https://en.wikipedia.org/wiki/Truncated_normal_distribution). Here a plot for $\lambda=\sigma=1$ enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.