Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

Filter by
Sorted by
Tagged with
37
votes
2answers
9k views

Is there any algorithm combining classification and regression?

I'm wondering if there's any algorithm could do classification and regression at the same time. For example, I'd like to let the algorithm learn a classifier, and at the same time within each label, ...
0
votes
2answers
448 views

PyMC3: Mixture Model with Latent Variables

I have a rather basic knowledge of Bayesian inference and I'm somewhat new to MCMC and PyMC3. Can I model data that looks like this? ...
1
vote
1answer
41 views

Defining the overlapping area of two log-normal distributions with different means, same variance, and different scaling factors that add up to 1

Define $$ \begin{cases} X_1\sim Lognormal(ln(\mu_1), \sigma^2) \\ X_2\sim Lognormal(ln(\mu_2), \sigma^2) \end{cases} $$ where $\mu_2>\mu_1>0$ and that there is a definite proportion, $\eta\in(0,...
0
votes
0answers
6 views

Importance sampling from a mixture model

Let us suppose to have the following mixture model $j \sim Cat(j|\pi)$, $x \sim p(x|j)$. Suppose that I observe a dataset of $\{j_i, x_i, f_i\}_{i=1}^N$, where $j_i \sim Cat(j|\pi), x_i \sim p(x_i|J_i)...
4
votes
1answer
53 views

Finite Beta mixture model in stan — mixture components not identified

I'm trying to model data $0 < Y_i < 1$ with a finite mixture of Beta components. To do this, I've adapted the code given in section 5.3 of the Stan manual. Instead of (log)normal priors, I am ...
1
vote
0answers
16 views

How to fit a statistical distribution data to a mixure of normal and lognormal distributions

I have a data set that is a mixture of one normal and one lognormal distribution. How can I fit the data to get the fit parameters? Can somebody help me with this.
1
vote
1answer
388 views

Calculation of AIC in finite mixture modeling

I have a question about calculation the AIC to find my optimal amount of clusters. I am applying mixture modeling with the EM algorithm. I know the formula AIC = -2ln(log-lik) + 2k. These are my log-...
3
votes
1answer
25 views

Reversible Jump for normal mixtures in R^d

I'm reading the article "Multivariate mixtures of normals with unknown number of components" (Dellaportas and Papageorgiou 2006). In this article they describe in great details how to ...
0
votes
2answers
202 views

Estimating two-component mixture of Weibull distributions?

Is there any existing package (preferably in Python or Matlab) to estimate the parameters of a two-component Weibull mixture model? And failing that, I am hoping to get some pointers towards rolling ...
6
votes
2answers
1k views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
0
votes
1answer
24 views

What is (are) scenerios and practical settings that can possibly lead to the weibull-log-Logistic mixture distribution?

In my paper I studied Weibull-loglogistic mixture distributions in reliability and life testing, some structural properties of the model are presented including moments, reliability, hazard rate ...
1
vote
1answer
167 views

Latent profile analysis - Gaussian models and which one to choose?

When performing LPA different models are possible depending on shape, volume and orientation (Figure 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5096736/). Can someone, please, explain in simple ...
0
votes
0answers
21 views

How to make inference on cluster-specific parameters in a Bayesian mixture model

Suppose I have a mixture model, for example of the kind $$ y_i \mid w, \{\theta_h\}, H \sim \sum_{h=1}^H w_h f(y_i \mid \theta_h) \\ P(H=h) = q_h \\ w \mid H \sim Dirichlet(\alpha) \\ \theta_1, \ldots,...
1
vote
1answer
203 views

Mixture model on binary + continuous data

If I have a dataset of continuous variables (that I can assume are normally distributed), I can identify subgroups using a Gaussian mixture model and implement. Likewise if I have binary data I can ...
0
votes
0answers
15 views

Mixture model for decomposing bimodal or multimodal distributions

The gaussian mixture model (GMM) is fed mixture components or features whose time series each have differing means and variances from one another, but are unimodal (have one mode) with each component ...
0
votes
0answers
9 views

Finite mixture model, nonlinear, parametric, individuals

Suppose that I think individual $i$ chooses number $c$ according to the formula $c = v(f(\delta, \gamma) x - \alpha y))$ where $v$ and $f$ are nonlinear functions. I know what form $v$ and $f$ take. I ...
0
votes
0answers
49 views

Mixture model for a mix of normal and lognormal distributions in R

I have a distribution composed of a mixture of a normal distribution and a log-normal distribution. A simulated example that looks quite similar to what I will expect in the real data would be: ...
0
votes
1answer
50 views

M step EM algorithm in Mixture Models. Expected value of the indicator variable under the posterior [closed]

I am not able to solve the following expectation. In the EM algorithm, the first step in the M step is to compute the expected value of $\log p(x,z)$ where $x$ are observations and $z$ indicator ...
0
votes
1answer
476 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
4
votes
1answer
81 views

Are Neural Networks Mixture Models?

To my understanding, Gaussian Mixture models are a set of parameterized gaussian distributions that collectively describe an entire, aggregate distribution. ^ from McGonagle et al Also to my ...
2
votes
0answers
16 views

Examples of mixture priors on SE-ARD length scale for Gaussian Process models?

Rasmussen and Williams (5.1) give the following notation for the SE-ARD kernel: $\begin{aligned}k(\mathbf{x}_p,\mathbf{x}_q)&=\sigma^2_f\hspace{0.5em} exp \left( -\frac{1}{2} (\mathbf{x}_p - \...
0
votes
0answers
8 views

Explaining Experimental data with EUT or PT: Using Structural Equation modeling or Fixed Mixture model?

I will try to be as clear as possible. Some background: I am doing experiments in economics and I have a huge dataset of experiments mirroring real life tax declaration. Participants in the lab ...
1
vote
1answer
24 views

Joint “density” of data and indicators in Bayesian mixture model

I'm currently working through the chapter on finite mixture models in BDA3 and came across the following model setup (with the usual slight abuse of notation): Let $\lambda=(\lambda_1,\dots,\lambda_H)...
1
vote
0answers
7 views

Should cluster intercepts add up to the overall mean in latent class clustering?

I am working with flexmix package in R to cluster my data (1341 p, 8 observations each). My DV is scaled per respondent, but the overall mean still remains 0. I am using a latent class regression ...
0
votes
0answers
17 views

Compute membership probabilities in E-step of EM algorithm with log-densities instead of densities

As an exercise I have implemented the EM algorithm for Gaussian mixtures, however, I have the problem that in high dimensions the densities of data points become so small that I get a numerical ...
2
votes
2answers
1k views

Splitting of bimodal distribution, use in regression models

I have a bimodal length-frequency distribution for the females of a species with a one-year life span. This pattern is not observed in the males. I suspect that the bimodality is due to different ...
0
votes
0answers
14 views

How do I correctly specify bayesian zero-inflated finite mixture model in jags?

I have basic understanding of Bayesian models and can fit simple models using "rjags". However, I am trying to fit a zero-inflated finite mixture model with n components (say n=2). I expect each ...
0
votes
0answers
132 views

EM algorithm for mixture of categorical distributions instantly stabilizes

Brief Summary of Question I'm trying to fit a mixture model of categorical distributions (see https://en.wikipedia.org/wiki/Categorical_distribution). The expectation at the second time step is ...
0
votes
0answers
11 views

How can I assess the number of cases needed to distinguish whether responses are from one of several probability distributions?

In my experiment, participants choose one of three options on every trial (the trials are 3x3 versions of the well known Prisoner's Dilemma game). What decision rule they use determines the ...
0
votes
0answers
10 views

Is this a typo in the `mclust` exposition?

I'm reading about the mclust package for gaussian mixture models. I want to understand its statistical assumptions. https://link.springer.com/content/pdf/10.1007/...
2
votes
0answers
350 views

Bayesian Nonparametric Latent feature model

For quite a long time I've been trying to understand the paper "Bayesian Nonparametric Latent feature model" (by Zoubin Ghahramani et al.) [http://mlg.eng.cam.ac.uk/zoubin/papers/GhaGriSol06.pdf]. In ...
1
vote
0answers
370 views

Generalised Linear Mixed Effect Models using latent class probabilites as weights

I'm relatively new to the field of generalised linear mixed models (GLMM) so this may be a redundant question but anyway... I'm trying to create a regression model to predict a binary outcome where I ...
1
vote
0answers
15 views

Conditional draws from a multivariate mixture model keeping one variable fixed

I would like to draw samples from a multivariate mixture model, for a given value of one of the variables. Assuming a gaussian mixture distribution built on $P_{1..p}$ variables with $K$ components: ...
1
vote
0answers
16 views

How to sample from a mixture of densities of transformed random variables?

Suppose we are given a set of $m$ random variables, $X_1$, $X_2$, ..., $X_m$, defined over the same set $\mathcal{X}$, with known densities $p_{X_i}$, for $i=1$, $2$, ..., $m$. Assume that getting ...
0
votes
0answers
379 views

Comparing 2 mixture models using mixtools

I have 2 mixture models I'd like to compare. Specifically, I want to compare lamda (i.e. proportion/area under each distribution) as it looks like there are differences there. Is this possible? ...
0
votes
1answer
428 views

How to fit nonparametric mixtures of regression models in a statistical program? [closed]

I am interested in estimating a nonparametric finite mixture regression model, explained for example here: https://methodology.psu.edu/media/techreports/09-93.pdf But I don't know with which program ...
3
votes
1answer
266 views

A Bernoulli mixture model with a Dirichlet prior on the parameters

Let's $x_1,...,x_N$ be a set of observation coming from the following generative process: $$ \boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
1
vote
1answer
34 views

How many parameters are present in a (general) discrete mixture of five normal distributions?

What is the minimal amount of parameters that can be retained in a particular context?
0
votes
0answers
28 views

Composition of Normals — Finite mixture with some known parameters

I.e., the data was generated from 5 normal distributions: ...
0
votes
1answer
32 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
1
vote
0answers
33 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
3
votes
2answers
326 views

Universal Approximation Capabilities of Mixture Models

I am looking for two reference incl. proofs showing 1) that a discrete Mixture of Gaussians can asymptotically approximate any (well behaved) continuous density 2) that a discrete Mixture of ...
4
votes
1answer
77 views

How to latently cluster regressors based on regressors' relationship with the outcome?

What is the best way/method to model patterns across coefficients and reduce number of coefficients based on these patterns? We have hundreds of regressors on the same scale and try to reduce the ...
1
vote
1answer
37 views

Is there a way to determine the number of the mixture components prior to run EM algorithm

I am working with mixture models. The common way to determine the number of the mixture components is fitting several mixture models with a different number of mixture components and then select the ...
1
vote
1answer
222 views

What is the relationship between gaussian mixture models and covariate regression?

Consider a histogram of observations $y$ which look a bit like this, Imagine that the two bell-curve shaped histograms (ignore the actual fitted curves) correspond to observations from females and ...
0
votes
0answers
72 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
8
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
2
votes
0answers
10 views

When can a finite mixture of distributions drawn from a distributional family be well-described by a distribution from the same family?

This question is motivated by a situation that we frequently encounter in economic variables. We believe that the overall distribution of some variable is well approximated by a particular ...
0
votes
1answer
31 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
0
votes
1answer
25 views

Equivalent way of rewriting a two-component mixture

I'm confused on the following equivalent way of rewriting a two-component mixture. Consider the two-component conditional mixture $$ F(z|x)=\lambda F_1(z|x)+(1-\lambda)F_2(z|x) $$ where all the $F$'...