Let there be observed data points $X = {X_1.. X_n .. X_N}$, where each $X_n \in R^D$. Lets assume these are distributed as a Gaussian $X \cong \mathcal N(\mu,\Sigma)$. Let us also assume that the mean has a Normal prior and covariance has Inverse Wishart prior. Since these are conjugate priors, given observed data $X$, we can draw posterior samples for mean and covariance using standard posterior update equations. What we have so far is just the usual Bayesian stuff.

Let the data points be formed from a union of two sets of data $X = {X^+} \cup {X^-}$. The data is still distributed as Gaussian. However lets assume that while drawing posteriors for mean and covariance, we have to give more weight to data points in ${X^+}$ than ${X^-}$, with the weight itself being some (hyper)parameter.

Note that we are NOT talking about mixture of gaussians. We dont say there are 2 mixtures and the data might have arised from one of these mixtures. We say the data still arises from ONE gaussian, with the mean and covariance primarily according to ${X^+}$, but perhaps altered(shifted?) by some proportion because of ${X^-}$ influence.

I am not sure how i can write the distribution for such a thing. I initially thought i can write $X \cong \mathcal N(A\mu^+ + B\mu^-,A\Sigma^+ + B\Sigma^-)$. We then draw $\mu^+, \Sigma^+$ from ${X^+}$ and similarly $\mu^-, \Sigma^-$ from ${X^-}$ by usual methods. We treat $A$ and $B$ as some (hyper)parameters. If i write like this, what is my assumption of the data. Am i implying $X = {X^+} + {X^-}$ rather than a union? Or the implication is too non-sensical to even worth a description?

Why would i want to do this - thats a lengthy explanation and would distract the specific question. The specific question is - "I have data partitioned into two sets. The data arises from a gaussian distribution. I want to draw posterior samples for the gaussian parameters according to one of the set. However the other set also has (marginal) influence on the parameters. How can I re-parametrize the distribution to account for this?"

Ps: Any approximate re-parametrization is fine. I've used the word "sampling", hopefully that wouldn't distract the specific question to ideas such as lets reject samples based on some function. Any references to the literature that does something similar is also welcome. Thanks in advance for suggestions.

  • 1
    $\begingroup$ I don't follow from very early on. How is generating posterior data standard Bayesian stuff? Bayesian analysis involves priors on parameters or hyper parameters a data set and then a resulting posterior distribution for the parameters. There is nothing that we would call posterior data. I guess since you have a posterior distribution for the mean and covariance you could draw "samples' for the parameters using the posterior distribution. But why would you need that? $\endgroup$ – Michael Chernick Sep 24 '12 at 20:32
  • $\begingroup$ You know the posterior distribution for the parameters so why would you draw inference on the parameters from that distribution. There seems to be no need to generate simulated data from the posterior distribution. $\endgroup$ – Michael Chernick Sep 24 '12 at 20:33
  • $\begingroup$ @Michael - I mean when your parameter's distribution has conjugate prior, then given observed data, it is straightforward to draw samples for the parameter posteriors. We would need to draw samples for the parameter to use it during inference. You draw samples for one set of parameters, use these samples to draw samples for another set of parameters and keep alternating between them (Gibbs Sampling). $\endgroup$ – Grafix1 Sep 25 '12 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.