Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

Mixture model for a mix of normal and lognormal distributions in R

I have a distribution composed of a mixture of a normal distribution and a log-normal distribution. A simulated example that looks quite similar to what I will expect in the real data would be: ...
3
votes
1answer
58 views

Are Neural Networks Mixture Models?

To my understanding, Gaussian Mixture models are a set of parameterized gaussian distributions that collectively describe an entire, aggregate distribution. ^ from McGonagle et al Also to my ...
0
votes
1answer
36 views

M step EM algorithm in Mixture Models. Expected value of the indicator variable under the posterior [closed]

I am not able to solve the following expectation. In the EM algorithm, the first step in the M step is to compute the expected value of $\log p(x,z)$ where $x$ are observations and $z$ indicator ...
2
votes
0answers
10 views

Examples of mixture priors on SE-ARD length scale for Gaussian Process models?

Rasmussen and Williams (5.1) give the following notation for the SE-ARD kernel: $\begin{aligned}k(\mathbf{x}_p,\mathbf{x}_q)&=\sigma^2_f\hspace{0.5em} exp \left( -\frac{1}{2} (\mathbf{x}_p - \...
0
votes
0answers
6 views

Explaining Experimental data with EUT or PT: Using Structural Equation modeling or Fixed Mixture model?

I will try to be as clear as possible. Some background: I am doing experiments in economics and I have a huge dataset of experiments mirroring real life tax declaration. Participants in the lab ...
1
vote
0answers
6 views

Should cluster intercepts add up to the overall mean in latent class clustering?

I am working with flexmix package in R to cluster my data (1341 p, 8 observations each). My DV is scaled per respondent, but the overall mean still remains 0. I am using a latent class regression ...
0
votes
0answers
10 views

Compute membership probabilities in E-step of EM algorithm with log-densities instead of densities

As an exercise I have implemented the EM algorithm for Gaussian mixtures, however, I have the problem that in high dimensions the densities of data points become so small that I get a numerical ...
0
votes
0answers
10 views

How do I correctly specify bayesian zero-inflated finite mixture model in jags?

I have basic understanding of Bayesian models and can fit simple models using "rjags". However, I am trying to fit a zero-inflated finite mixture model with n components (say n=2). I expect each ...
0
votes
0answers
48 views

EM algorithm for mixture of categorical distributions instantly stabilizes

Brief Summary of Question I'm trying to fit a mixture model of categorical distributions (see https://en.wikipedia.org/wiki/Categorical_distribution). The expectation at the second time step is ...
0
votes
0answers
10 views

How can I assess the number of cases needed to distinguish whether responses are from one of several probability distributions?

In my experiment, participants choose one of three options on every trial (the trials are 3x3 versions of the well known Prisoner's Dilemma game). What decision rule they use determines the ...
0
votes
0answers
8 views

Is this a typo in the `mclust` exposition?

I'm reading about the mclust package for gaussian mixture models. I want to understand its statistical assumptions. https://link.springer.com/content/pdf/10.1007/...
1
vote
0answers
143 views

Generalised Linear Mixed Effect Models using latent class probabilites as weights

I'm relatively new to the field of generalised linear mixed models (GLMM) so this may be a redundant question but anyway... I'm trying to create a regression model to predict a binary outcome where I ...
1
vote
0answers
128 views

Bayesian Nonparametric Latent feature model

For quite a long time I've been trying to understand the paper "Bayesian Nonparametric Latent feature model" (by Zoubin Ghahramani et al.) [http://mlg.eng.cam.ac.uk/zoubin/papers/GhaGriSol06.pdf]. In ...
1
vote
0answers
11 views

Conditional draws from a multivariate mixture model keeping one variable fixed

I would like to draw samples from a multivariate mixture model, for a given value of one of the variables. Assuming a gaussian mixture distribution built on $P_{1..p}$ variables with $K$ components: ...
1
vote
0answers
14 views

How to sample from a mixture of densities of transformed random variables?

Suppose we are given a set of $m$ random variables, $X_1$, $X_2$, ..., $X_m$, defined over the same set $\mathcal{X}$, with known densities $p_{X_i}$, for $i=1$, $2$, ..., $m$. Assume that getting ...
0
votes
0answers
149 views

Comparing 2 mixture models using mixtools

I have 2 mixture models I'd like to compare. Specifically, I want to compare lamda (i.e. proportion/area under each distribution) as it looks like there are differences there. Is this possible? ...
1
vote
1answer
152 views

Calculation of AIC in finite mixture modeling

I have a question about calculation the AIC to find my optimal amount of clusters. I am applying mixture modeling with the EM algorithm. I know the formula AIC = -2ln(log-lik) + 2k. These are my log-...
0
votes
0answers
185 views

Difference between GMM and HMM

From what I understand: GMM is a probabilistic model which can model N sub population normally distributed. Each component in GMM is a Gaussian distribution. HMM is a statistical Markov model with ...
3
votes
1answer
235 views

A Bernoulli mixture model with a Dirichlet prior on the parameters

Let's $x_1,...,x_N$ be a set of observation coming from the following generative process: $$ \boldsymbol{\theta} \sim \text{Dirichlet}(\boldsymbol{\alpha})\qquad\boldsymbol{\theta},\boldsymbol\alpha\...
2
votes
2answers
369 views

Splitting of bimodal distribution, use in regression models

I have a bimodal length-frequency distribution for the females of a species with a one-year life span. This pattern is not observed in the males. I suspect that the bimodality is due to different ...
0
votes
1answer
28 views

How many parameters are present in a (general) discrete mixture of five normal distributions?

What is the minimal amount of parameters that can be retained in a particular context?
0
votes
0answers
31 views

What is the issue when label switching happens?

While studying mixture model, I met label switching problem. I couldn't see the problematic issue of label switching. For me, when label switching happens, we are still able to use the model to ...
0
votes
0answers
125 views

identify random-choosers in discrete choice data using gmnl

I read (see sources below) that one can identify "random-choosers" (or mischievous respondents) with finite mixture models/latent class analysis as a seperate class. As far as I undersand it this can ...
0
votes
0answers
22 views

Concomitant Variables in Finite mixture Models

On what basis does one decide whether a variable is to be taken as a concomitant variable in a finite mixture model (using flexmix in R)? With experimental data this seems easy to answer, but what ...
1
vote
1answer
100 views

Mixture model on binary + continuous data

If I have a dataset of continuous variables (that I can assume are normally distributed), I can identify subgroups using a Gaussian mixture model and implement. Likewise if I have binary data I can ...
0
votes
1answer
29 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
1
vote
0answers
76 views

Closed form for Finite Gaussian Mixture Model when weights are known and prior variance can be 0

Suppose I have a normal likelihood $x|\theta \sim N(\theta, \sigma^2_{known})$ where the variance is known and a mixture prior $\theta \sim p * N(\mu_1, \sigma^2_1) + (1-p) * N(\mu_2, \sigma^2_2)$, ...
1
vote
0answers
31 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
4
votes
1answer
76 views

How to latently cluster regressors based on regressors' relationship with the outcome?

What is the best way/method to model patterns across coefficients and reduce number of coefficients based on these patterns? We have hundreds of regressors on the same scale and try to reduce the ...
0
votes
1answer
30 views

Is there a way to determine the number of the mixture components prior to run EM algorithm

I am working with mixture models. The common way to determine the number of the mixture components is fitting several mixture models with a different number of mixture components and then select the ...
0
votes
1answer
114 views

Estimating two-component mixture of Weibull distributions?

Is there any existing package (preferably in Python or Matlab) to estimate the parameters of a two-component Weibull mixture model? And failing that, I am hoping to get some pointers towards rolling ...
0
votes
0answers
19 views

Data distributions extrapolation

Is there a way to extrapolate (according to variance or other statistics) distributions from data? In other words, I have a gene expression matrix in which rows are genes while columns are samples (i....
0
votes
0answers
64 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
1
vote
1answer
110 views

Latent profile analysis - Gaussian models and which one to choose?

When performing LPA different models are possible depending on shape, volume and orientation (Figure 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5096736/). Can someone, please, explain in simple ...
2
votes
0answers
8 views

When can a finite mixture of distributions drawn from a distributional family be well-described by a distribution from the same family?

This question is motivated by a situation that we frequently encounter in economic variables. We believe that the overall distribution of some variable is well approximated by a particular ...
1
vote
1answer
21 views

Joint “density” of data and indicators in Bayesian mixture model

I'm currently working through the chapter on finite mixture models in BDA3 and came across the following model setup (with the usual slight abuse of notation): Let $\lambda=(\lambda_1,\dots,\lambda_H)...
0
votes
1answer
24 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
2
votes
1answer
29 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
3
votes
1answer
30 views

Is it acceptable in mathematical saying that the E-step is equal to posterior probability

I am studying EM-algorithm for mixture data. I read that some authors said that, the E-step is equivalent to the calculation of the posterior probability (I think this come from Bayesian rule). So, my ...
0
votes
1answer
24 views

Equivalent way of rewriting a two-component mixture

I'm confused on the following equivalent way of rewriting a two-component mixture. Consider the two-component conditional mixture $$ F(z|x)=\lambda F_1(z|x)+(1-\lambda)F_2(z|x) $$ where all the $F$'...
1
vote
0answers
26 views

A 2 component mixture is symmetric if and only if $\lambda\in \{0,1,\frac{1}{2}\}$

Consider the following mixture of two densities $$ f(x)=\lambda g(x-\mu_1)+(1-\lambda)g(x-\mu_2) $$ with $\lambda\in [0,1]$, $g(\cdot)$ symmetric around zero, $\mu_1<\mu_2$. Claim: the mixture is ...
0
votes
1answer
43 views

Mixture of two components

Consider a univariate probability density function $p(x)$ that is a mixture of $2$ probability density functions with weights $\eta, 1-\eta$ and $\eta\in (0,1)$: $$ p(x)=(1-\eta)g(x)+\eta f(x) \...
2
votes
1answer
125 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
353 views

Posterior distribution of mixture models

In the context of mixture models in Bayesian inference, one can assume that the general form of the joint posterior for a mixture model of $k$ components is $$ \begin{equation} p( \boldsymbol{\...
3
votes
2answers
64 views

Confusion in modelling finite mixture model

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated: The usual representation (of a finite ...
3
votes
1answer
90 views

Generate a finite mixture distribution by sampling mixture component parameters

Assume I have a some mixture distribution, $H$, with mean $\mu$ and variance $\sigma^2$. $H$ is a mixture of $n$ component distributions where all component weights are equal. Let $\mu_i$ be the mean ...
1
vote
1answer
140 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
0
votes
1answer
363 views

PyMC3: Mixture Model with Latent Variables

I have a rather basic knowledge of Bayesian inference and I'm somewhat new to MCMC and PyMC3. Can I model data that looks like this? ...
2
votes
1answer
67 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
0
votes
0answers
295 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...