2
$\begingroup$

If I have a dataset of continuous variables (that I can assume are normally distributed), I can identify subgroups using a Gaussian mixture model and implement. Likewise if I have binary data I can model it as a mixture of Bernoulli.

I can implement either of these models using likelihood EM approach, or MCMC (typically Gibbs) sampling.

However, if my dataset comprises both continuous and binary data, can a mixture model be defined? If so, how?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Yes, it can. I shall expand into an answer later. However, why would you not be able to define a mixture model? Consider the likelihood function for one person in either type of model: it's the product of all the probabilities that their indicators were equal to whatever was observed. Why can't this adapt to mixed types of indicators? Also see this answer: stats.stackexchange.com/questions/339936/… $\endgroup$
    – Weiwen Ng
    Sep 10, 2019 at 16:45
  • $\begingroup$ I hadn't seen it done before or implemented in any general probabilistic programming frameworks. Do you know if any existing software can infer such models or do custom Gibbs samplers need to be written? $\endgroup$
    – Stuart Lacy
    Sep 11, 2019 at 20:22
  • 1
    $\begingroup$ I'm unfamiliar with Bayesian implementations. I have fit a model in Stata with mixed types of indicators. I am almost sure that MPlus can do this also. I believe that the R package flexmix does; its function FLXMVcombi says that "This model driver can be used to cluster mixed-mode binary and Gaussian data. It checks which columns of a matrix contain only zero and ones, and does the same as FLXMCmvbinary for them. For the remaining columns of the data matrix independent Gaussian distributions are used ..." $\endgroup$
    – Weiwen Ng
    Sep 11, 2019 at 21:08
  • 1
    $\begingroup$ Ah great, I didn't realise it was quite as straight forward as simply incorporating the different data type into the likelihood product. I'll check out that flexmix implementation, thanks! $\endgroup$
    – Stuart Lacy
    Sep 16, 2019 at 10:32

1 Answer 1

Reset to default
1
$\begingroup$

Yes, you can simply combine the two components of you make a simple independence assumption.

Then P(continuous, binary) = P(continuous) P(binary) and you already have the solution of the right hand side.

For binary variables I don't think you want to use a dependent model here, but it is equally easy to do for at least one or two variables.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.