Updating conjugate priors with noisy observations

Question

I'm considering a problem that has been partially addressed elsewhere: Bayesian updating with conjugate priors using the closed form expressions

but now I have an added twist. My samples are drawn from distributions of unknown mean and variance, and I wish to estimate this mean and variance. However, instead of direct access to the samples, I only have noisy observations of them, which I model by addition of Gaussian noise of ~N(0,s) to the each sample independently, and s is known. How do I incorporate this additional noise term to the posterior worked out for the original problem above?

Answer 1

If the samples follow a normal distribution then the added noise simply adds the knowns to the variance of the samples. So this affects the likelihood function. Then the posterior can be computed in the usual way. If you are assuming a non-normal distribution, things are complicated because you need to determine the distribution of the observation which is now the sample and the independent Gaussian noise term.

Answer 2

You cannot because you do not have an additional variable correlated either with the noise or the true value of $\sigma$ for the underlying.

Let me give you an example. Imagine that you had perfect measurements when you started your experiment, but at some point the machine doing the measurement broke. The problem was that it added noise, but did not impact the center of location. Again, for simplicity, we will assume normality throughout.

Your problem would include $\sigma_{true}$, $\sigma_{noise}$, and a parameter $k$ which is an estimate of the timing of the break.

To the left of the partition you would estimate $\mathcal{N}(\mu,\sigma_{true}^2)$. To the right of the partition you would model $\mathcal{N}(\mu,\sigma_{true}^2+\sigma_{noise}^2)$ and your partition would follow some distribution as well. There are several candidate distributions depending on how you want to model where the break happens.

There is no conjugacy based solution, but without a to distinguish, the most you know is that $\sigma_{true}\le\sigma_{measured}$.

Without a solution to filter the results, you have an ordinary conjugacy problem. It might be important to note that as Bayesian probability is $\Pr(\mu;\sigma|X)$, where $X$ is data, it treats data as fixed. Randomness does not exist, it is rather just uncertainty as to the transform of the real data.

You are arguing that $X_{observed}=f(X_{true})$, but that will require some way to measure $f$ with no additional source of data, which cannot be done.

Your problem is that $\sigma_{observed}^2=\sigma_{true}^2+\sigma_{noise}^2.$ This is no different than the generic problem $c=a+b$, with $a;b>0$ There are an infinite number of solutions to the set of a and b.

It doesn't upset the conjugacy calculation because you cannot differentiate the two. If you could differentiate signal and noise, even weakly, conjugacy would not apply and you would have to do numerical integration.