Combining Probability Density Functions I am trying to predict the outcome of a random variable x, which is a real-valued number. In some cases I can observe another variable y1, which should approximate x. I model y1 as a Gaussian distribution with mean of 0 and an empirically estimated standard deviation, and use the probability density function of that Gaussian to predict x.
In some cases I also have a second observation y2, which I can model similarly as a Gaussian.
What is the appropriate way to combine y1 and y2 into an estimate on x? Should I add the distributions and model x as a mixture of Gaussians, or should I multiply them? Or, should I try both and pick the answer that maximizes the probability of sampling an independent set of data?
 A: I can tell you what I would do as a machine learner ;):


*

*Creating two models $M_1,M_2$ for x using $y_1,y_2$ respectively

*A prediction for x is calculated as the average of the predictions of $M_1$ and $M_2$


A "model" can be either a gaussian distribution (as described, if you know that x a) has a gaussian distribution and b) know that the function x to y is nearly the identity) or anything else, e.g. a simple linear regression model (if you are not sure whether there are additional factors).
BUT: If x is indeed a bimodal distribution with the two gaussians y, than the suggested approach would not make any sense. In this case I'd try a more generic approach, i.e. EM to see whether one or two gaussians are more appropriate.
It is quite hard to give a general answer to this questions, since it is not exactly clear what determines whether y1 or y2 can be observed. It could be that for both y x is missing completely at random (in this case y1 and y2 were just different random samples), but on the other hand it could be that for a certain fraction of x only y1 can be observed, but not y2, and vice versa for another fraction of x.
