Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

Filter by
Sorted by
Tagged with
1
vote
1answer
39 views

Defining the overlapping area of two log-normal distributions with different means, same variance, and different scaling factors that add up to 1

Define $$ \begin{cases} X_1\sim Lognormal(ln(\mu_1), \sigma^2) \\ X_2\sim Lognormal(ln(\mu_2), \sigma^2) \end{cases} $$ where $\mu_2>\mu_1>0$ and that there is a definite proportion, $\eta\in(0,...
0
votes
0answers
5 views

Importance sampling from a mixture model

Let us suppose to have the following mixture model $j \sim Cat(j|\pi)$, $x \sim p(x|j)$. Suppose that I observe a dataset of $\{j_i, x_i, f_i\}_{i=1}^N$, where $j_i \sim Cat(j|\pi), x_i \sim p(x_i|J_i)...
3
votes
1answer
44 views

Finite Beta mixture model in stan — mixture components not identified

I'm trying to model data $0 < Y_i < 1$ with a finite mixture of Beta components. To do this, I've adapted the code given in section 5.3 of the Stan manual. Instead of (log)normal priors, I am ...
1
vote
0answers
14 views

How to fit a statistical distribution data to a mixure of normal and lognormal distributions

I have a data set that is a mixture of one normal and one lognormal distribution. How can I fit the data to get the fit parameters? Can somebody help me with this.
3
votes
1answer
25 views

Reversible Jump for normal mixtures in R^d

I'm reading the article "Multivariate mixtures of normals with unknown number of components" (Dellaportas and Papageorgiou 2006). In this article they describe in great details how to ...
0
votes
1answer
24 views

What is (are) scenerios and practical settings that can possibly lead to the weibull-log-Logistic mixture distribution?

In my paper I studied Weibull-loglogistic mixture distributions in reliability and life testing, some structural properties of the model are presented including moments, reliability, hazard rate ...
0
votes
0answers
21 views

How to make inference on cluster-specific parameters in a Bayesian mixture model

Suppose I have a mixture model, for example of the kind $$ y_i \mid w, \{\theta_h\}, H \sim \sum_{h=1}^H w_h f(y_i \mid \theta_h) \\ P(H=h) = q_h \\ w \mid H \sim Dirichlet(\alpha) \\ \theta_1, \ldots,...
0
votes
0answers
11 views

Mixture model for decomposing bimodal or multimodal distributions

The gaussian mixture model (GMM) is fed mixture components or features whose time series each have differing means and variances from one another, but are unimodal (have one mode) with each component ...
0
votes
0answers
9 views

Finite mixture model, nonlinear, parametric, individuals

Suppose that I think individual $i$ chooses number $c$ according to the formula $c = v(f(\delta, \gamma) x - \alpha y))$ where $v$ and $f$ are nonlinear functions. I know what form $v$ and $f$ take. I ...
0
votes
0answers
42 views

Mixture model for a mix of normal and lognormal distributions in R

I have a distribution composed of a mixture of a normal distribution and a log-normal distribution. A simulated example that looks quite similar to what I will expect in the real data would be: ...
4
votes
1answer
80 views

Are Neural Networks Mixture Models?

To my understanding, Gaussian Mixture models are a set of parameterized gaussian distributions that collectively describe an entire, aggregate distribution. ^ from McGonagle et al Also to my ...
0
votes
1answer
46 views

M step EM algorithm in Mixture Models. Expected value of the indicator variable under the posterior [closed]

I am not able to solve the following expectation. In the EM algorithm, the first step in the M step is to compute the expected value of $\log p(x,z)$ where $x$ are observations and $z$ indicator ...
2
votes
0answers
12 views

Examples of mixture priors on SE-ARD length scale for Gaussian Process models?

Rasmussen and Williams (5.1) give the following notation for the SE-ARD kernel: $\begin{aligned}k(\mathbf{x}_p,\mathbf{x}_q)&=\sigma^2_f\hspace{0.5em} exp \left( -\frac{1}{2} (\mathbf{x}_p - \...
0
votes
0answers
8 views

Explaining Experimental data with EUT or PT: Using Structural Equation modeling or Fixed Mixture model?

I will try to be as clear as possible. Some background: I am doing experiments in economics and I have a huge dataset of experiments mirroring real life tax declaration. Participants in the lab ...
1
vote
0answers
7 views

Should cluster intercepts add up to the overall mean in latent class clustering?

I am working with flexmix package in R to cluster my data (1341 p, 8 observations each). My DV is scaled per respondent, but the overall mean still remains 0. I am using a latent class regression ...
0
votes
0answers
14 views

Compute membership probabilities in E-step of EM algorithm with log-densities instead of densities

As an exercise I have implemented the EM algorithm for Gaussian mixtures, however, I have the problem that in high dimensions the densities of data points become so small that I get a numerical ...
0
votes
0answers
13 views

How do I correctly specify bayesian zero-inflated finite mixture model in jags?

I have basic understanding of Bayesian models and can fit simple models using "rjags". However, I am trying to fit a zero-inflated finite mixture model with n components (say n=2). I expect each ...
0
votes
0answers
100 views

EM algorithm for mixture of categorical distributions instantly stabilizes

Brief Summary of Question I'm trying to fit a mixture model of categorical distributions (see https://en.wikipedia.org/wiki/Categorical_distribution). The expectation at the second time step is ...
0
votes
0answers
11 views

How can I assess the number of cases needed to distinguish whether responses are from one of several probability distributions?

In my experiment, participants choose one of three options on every trial (the trials are 3x3 versions of the well known Prisoner's Dilemma game). What decision rule they use determines the ...
0
votes
0answers
10 views

Is this a typo in the `mclust` exposition?

I'm reading about the mclust package for gaussian mixture models. I want to understand its statistical assumptions. https://link.springer.com/content/pdf/10.1007/...
1
vote
0answers
328 views

Generalised Linear Mixed Effect Models using latent class probabilites as weights

I'm relatively new to the field of generalised linear mixed models (GLMM) so this may be a redundant question but anyway... I'm trying to create a regression model to predict a binary outcome where I ...
1
vote
0answers
309 views

Bayesian Nonparametric Latent feature model

For quite a long time I've been trying to understand the paper "Bayesian Nonparametric Latent feature model" (by Zoubin Ghahramani et al.) [http://mlg.eng.cam.ac.uk/zoubin/papers/GhaGriSol06.pdf]. In ...
1
vote
0answers
15 views

Conditional draws from a multivariate mixture model keeping one variable fixed

I would like to draw samples from a multivariate mixture model, for a given value of one of the variables. Assuming a gaussian mixture distribution built on $P_{1..p}$ variables with $K$ components: ...
1
vote
0answers
16 views

How to sample from a mixture of densities of transformed random variables?

Suppose we are given a set of $m$ random variables, $X_1$, $X_2$, ..., $X_m$, defined over the same set $\mathcal{X}$, with known densities $p_{X_i}$, for $i=1$, $2$, ..., $m$. Assume that getting ...
0
votes
0answers
336 views

Comparing 2 mixture models using mixtools

I have 2 mixture models I'd like to compare. Specifically, I want to compare lamda (i.e. proportion/area under each distribution) as it looks like there are differences there. Is this possible? ...
1
vote
1answer
341 views

Calculation of AIC in finite mixture modeling

I have a question about calculation the AIC to find my optimal amount of clusters. I am applying mixture modeling with the EM algorithm. I know the formula AIC = -2ln(log-lik) + 2k. These are my log-...
0
votes
0answers
285 views

Difference between GMM and HMM

From what I understand: GMM is a probabilistic model which can model N sub population normally distributed. Each component in GMM is a Gaussian distribution. HMM is a statistical Markov model with ...
3
votes
1answer
261 views

A Bernoulli mixture model with a Dirichlet prior on the parameters

Let's $x_1,...,x_N$ be a set of observation coming from the following generative process: $$ \boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
2
votes
2answers
940 views

Splitting of bimodal distribution, use in regression models

I have a bimodal length-frequency distribution for the females of a species with a one-year life span. This pattern is not observed in the males. I suspect that the bimodality is due to different ...
1
vote
1answer
31 views

How many parameters are present in a (general) discrete mixture of five normal distributions?

What is the minimal amount of parameters that can be retained in a particular context?
1
vote
1answer
178 views

Mixture model on binary + continuous data

If I have a dataset of continuous variables (that I can assume are normally distributed), I can identify subgroups using a Gaussian mixture model and implement. Likewise if I have binary data I can ...
0
votes
1answer
32 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
1
vote
0answers
32 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
4
votes
1answer
77 views

How to latently cluster regressors based on regressors' relationship with the outcome?

What is the best way/method to model patterns across coefficients and reduce number of coefficients based on these patterns? We have hundreds of regressors on the same scale and try to reduce the ...
1
vote
1answer
35 views

Is there a way to determine the number of the mixture components prior to run EM algorithm

I am working with mixture models. The common way to determine the number of the mixture components is fitting several mixture models with a different number of mixture components and then select the ...
0
votes
2answers
180 views

Estimating two-component mixture of Weibull distributions?

Is there any existing package (preferably in Python or Matlab) to estimate the parameters of a two-component Weibull mixture model? And failing that, I am hoping to get some pointers towards rolling ...
0
votes
0answers
69 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
1
vote
1answer
156 views

Latent profile analysis - Gaussian models and which one to choose?

When performing LPA different models are possible depending on shape, volume and orientation (Figure 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5096736/). Can someone, please, explain in simple ...
2
votes
0answers
10 views

When can a finite mixture of distributions drawn from a distributional family be well-described by a distribution from the same family?

This question is motivated by a situation that we frequently encounter in economic variables. We believe that the overall distribution of some variable is well approximated by a particular ...
1
vote
1answer
23 views

Joint “density” of data and indicators in Bayesian mixture model

I'm currently working through the chapter on finite mixture models in BDA3 and came across the following model setup (with the usual slight abuse of notation): Let $\lambda=(\lambda_1,\dots,\lambda_H)...
0
votes
1answer
30 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
2
votes
1answer
30 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
3
votes
1answer
33 views

Is it acceptable in mathematical saying that the E-step is equal to posterior probability

I am studying EM-algorithm for mixture data. I read that some authors said that, the E-step is equivalent to the calculation of the posterior probability (I think this come from Bayesian rule). So, my ...
0
votes
1answer
25 views

Equivalent way of rewriting a two-component mixture

I'm confused on the following equivalent way of rewriting a two-component mixture. Consider the two-component conditional mixture $$ F(z|x)=\lambda F_1(z|x)+(1-\lambda)F_2(z|x) $$ where all the $F$'...
1
vote
0answers
26 views

A 2 component mixture is symmetric if and only if $\lambda\in \{0,1,\frac{1}{2}\}$

Consider the following mixture of two densities $$ f(x)=\lambda g(x-\mu_1)+(1-\lambda)g(x-\mu_2) $$ with $\lambda\in [0,1]$, $g(\cdot)$ symmetric around zero, $\mu_1<\mu_2$. Claim: the mixture is ...
0
votes
1answer
43 views

Mixture of two components

Consider a univariate probability density function $p(x)$ that is a mixture of $2$ probability density functions with weights $\eta, 1-\eta$ and $\eta\in (0,1)$: $$ p(x)=(1-\eta)g(x)+\eta f(x) \...
2
votes
1answer
142 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
393 views

Posterior distribution of mixture models

In the context of mixture models in Bayesian inference, one can assume that the general form of the joint posterior for a mixture model of $k$ components is $$ \begin{equation} p( \boldsymbol{\...
3
votes
2answers
68 views

Confusion in modelling finite mixture model

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated: The usual representation (of a finite ...
3
votes
1answer
94 views

Generate a finite mixture distribution by sampling mixture component parameters

Assume I have a some mixture distribution, $H$, with mean $\mu$ and variance $\sigma^2$. $H$ is a mixture of $n$ component distributions where all component weights are equal. Let $\mu_i$ be the mean ...