I am new to Bayesian inference so I apologise in advance if this seems a basic or poorly described question.

I am trying to perform the fusion of probabilities of multiple data points. To do this I am trying to formulate how I can do this with Bayesian inference. I understand how Bayesian inference can obtain a probability for two data points on the same distribution.

However, I now have two separate datasets of priors for two separate sources. I am given to understand that Bayesian inference must be conducted with independent and identically distributed variables. The two sets of priors in my case are not identically distributed.

Can I still perform Bayesian inference? I have considered a normalisation approach to get them on the same distribution but this seems like a poor solution.

  • 2
    $\begingroup$ Ignoring the math and statistics, how would you explain what you were trying to do to someone and what would success look like? $\endgroup$ – Dave Harris Apr 29 '18 at 2:53
  • 3
    $\begingroup$ Bayesian inference does not require independent and identically distributed variables. It is really unclear what your objective is and what data etc. you have; perhaps you could clarify? $\endgroup$ – jbowman Apr 29 '18 at 3:00
  • $\begingroup$ @jbowman I essentially have two distributions of data that represent two separate prior sensor measurements when a hypothesis is true. The distributions are different for each sensor. Given a future sample from each sensor I am trying to work out the belief in the two sensor scores given the prior readings. $\endgroup$ – mino Apr 29 '18 at 12:37

Let us assume we have some unobserved variable $\theta$ that represents a state, e.g., $\theta \in \{0,1\}$. We have two sensors, label them $s_1$ and $s_2$, and each produces a random measurement $x_i, i\in \{1,2\}$ that is dependent upon the state, with possibly different probability distributions $p_i(x_i;\theta)$. We'll further assume the random parts of the measurements are independent of each other.

Now, if $\theta = 0$, the probability distribution for the measurements $(x_1, x_2)$ is just the product of the sensor-specific probability distributions conditional upon $\theta=0$ (thanks to that independence assumption made in the previous sentence):

$$p(x_1,x_2; \theta = 0) = p_1(x_1;\theta=0)p_2(x_2;\theta=0)$$

and similarly for $\theta=1$. If we have a prior distribution on $\theta$, say $P(\theta)$, we can form our posterior distribution in the usual Bayesian way:

$$p(\theta;x_1,x_2) \propto p(x_1,x_2;\theta)P(\theta) = p_1(x_1;\theta)p_2(x_2;\theta)P(\theta)$$

ETA: For concreteness, let us assume we are monitoring a piece of equipment that produces power, e.g., a solar panel that tracks the sun. Something happens that may have damaged it, in which case it would a) produce less power, and b) consume more power trying to track the sun. So we have two different sensors reporting measurements, the sensor $s_1$ that tracks power production and the sensor $s_2$ that tracks power consumption.

If the tracker is good, power production ($x_1$) is distributed $\text{Normal}(100,10)$. If it is bad, power production is distributed $\text{Normal}(90,10)$.

If the tracker is good, power consumption ($x_2$) is distributed $\text{Gamma}(\mu=5,\sigma=2)$. If it is bad, power consumption is distributed $\text{Gamma}(\mu=10,\sigma=3)$. I am parameterizing the Gamma using the mean and standard deviation respectively for ease of interpreting the numbers.

If we define a parameter $\theta$ which equals 1 if the tracker is good and 0 if it is bad, we can write the probability distributions for $x_1$ and $x_2$ as follows:

$$p_1(x_1;\theta) = \text{Normal}(90 + 10\theta, 10)$$ $$p_2(x_2;\theta) = \text{Gamma}(10-5\theta,3-\theta)$$

Our prior probability is on whether or not the tracker is good, i.e., on $\theta$. We doubt it was damaged; let's set $P(\theta = 1) = 0.75$.

Now we look at our sensor readings. $x_1 = 97$ and $x_2 = 6.7$. On to the Bayesian calculation engine! The likelihood function for $\theta$ is just the product of the individual probability distributions $p_1$ and $p_2$:

$$L(\theta; x_1,x_2) = p_1(x_1;\theta)p_2(x_2;\theta)$$

and some calculation gives us the following results:

$$L(\theta=0; x_1=97, x_2=6.7) = 0.00281$$ $$L(\theta=1; x_1=97, x_2=6.7) = 0.00364$$

and our posterior probability that the tracker is good is:

$$P(\theta = 1) = {0.00364*0.75 \over 0.00364*0.75 + 0.00281*0.25} = 0.795$$

Note that it is not, strictly speaking, necessary that we construct a numeric parameter $\theta$ for this example. We could have just done everything using "Good" and "Bad" as conditions for selecting, in the case of sensor 1, which of two Normal distributions to use in the calculation, and similarly for sensor 2, and we would have gotten the same result.

| cite | improve this answer | |
  • $\begingroup$ Thank you for your explanation! The one this I am still a little lost on is how we deal with the fact that our x values for different sensors may have different probability distributions? You say we have a prior distribution $P(\theta)$. In practice though we will have a prior for each sensor. Therefore don't we need to combine the two priors? The priors for the x values for the current sensors have different means and standard deviations. $\endgroup$ – mino Apr 29 '18 at 19:08
  • $\begingroup$ Well, in the example I gave above, they do have different probability distributions. One has probability distribution $p_1$ and the other has probability distribution $p_2$; they can be completely different. Furthermore, your prior isn't on data from the sensor, it's on some parameter your sensors are giving you information about. If that parameter is different between the two sensors, then you have two completely separate problems. If the parameter is the same, and you have measurements from two different sensors, then the parameter is just $\theta$ above. $\endgroup$ – jbowman Apr 29 '18 at 19:16
  • $\begingroup$ Ah, I think I understand that! Thank you for the explanation! $\endgroup$ – mino Apr 29 '18 at 19:34
  • $\begingroup$ Does this mean that given the parameter is the same, the prior scores from both sensors form the prior $P(\theta)$ even if they have different means, etc? $\endgroup$ – mino Apr 29 '18 at 19:36
  • $\begingroup$ Let me expand the answer with an actual example, just to make things clearer. It'll take a little while to work it out. $\endgroup$ – jbowman Apr 29 '18 at 19:37

Not the answer you're looking for? Browse other questions tagged or ask your own question.