1
$\begingroup$

Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?

Recently, I came across the use of Gaussian Mixture Distributions being used to model the response variable in a regression setting:

enter image description here

If I understand the above correctly - this approach should allow for greater flexibility when the response variable in a regression problem appears to follow an irregular distribution. Thus, is such an approach the current "gold standard" for dealing with irregular distributions in regression problems?

Thanks!

References:

$\endgroup$
4
  • 2
    $\begingroup$ Can you clarify what you mean by an "irregular distribution?" i.e., what makes it irregular? I've seen mixture models for some clustering applications. However, since I work in machine learning, in certain cases (without implying that this is right or wrong), neural networks have been used for their function approximation capabilities (that guarantee comes with a ton of caveats, but empirically can work out). This is an extremely general option though. $\endgroup$
    – tchainzzz
    Nov 24 at 6:08
  • $\begingroup$ @ tchainzzz: Thank you for your reply! By irregular, I meant that Mixture Normal Distributions can fit custom distributions that capture patterns in some specific response variable - that other probability distributions poorly model (e.g. poisson, normal). $\endgroup$
    – stats555
    Nov 24 at 6:27
  • $\begingroup$ blogs.sas.com/content/iml/2011/10/21/… $\endgroup$
    – stats555
    Nov 24 at 6:27
  • $\begingroup$ I can't speak to whether this is the "standard approach" in general (and this might be field-dependent as well). But if you have a reason to believe that your data is best described in terms of some "clusters" inherent to the data distribution (i.e. a bimodal distribution), where each cluster can be described in some distribution, then a mixture model seems like a decent baseline? I wonder if this is simply a case where you'll have to just try it out and see. $\endgroup$
    – tchainzzz
    Nov 24 at 6:31
1
$\begingroup$

Mixture distributions make the most sense if we have grounds to suspect that the data belong to multiple different subpopulations, as per the textbook you cite. Usually, we don't know which of the subpopulation a given instance belongs to - and we may not even know how many such subpopulations exist.

As such, we could even consider straightforward ANOVA as a kind of mixture model, where we are only interested in the subpopulation means, and where we already know which subpopulation a given instance belongs to.

Another common example of mixture models are zero-inflated and related (e.g., hurdle) models/distributions. The idea is that there are two underlying processes, one resulting in a zero observation and the other in some other observation (which may be zero itself, or constrained to be non-zero). For instance, you could model supermarket sales that way: either nobody demanded the product, or someone did demand it and then possibly bought multiple units.

Then again, there are many situations where non-mixture models are appropriate, but rather things like GLMs. If you want to model something that is inherently positive, you can use gamma regression. You can use negative binomial regression for those supermarket sales. Are gamma or negbin distributions already "irregular"? I wouldn't say so.

So, to summarize, I would say that mixtures are good precisely when your data exhibits a specific kind of regularity, namely hypothesized subpopulations. It's definitely not the go-to tool if you have "just any kind of irregularity". (What do you do with "irregular" data? You try to understand it better to find out which tool is appropriate.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.