Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

56 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
2answers
1k views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
6
votes
0answers
922 views

Population Monte Carlo Algorithm

I am trying to wrap my head around the Population Monte Carlo Algorithm. I want to implement it for a mixture model, but I am uncertain on how to proceed. I am mostly looking for references or ...
3
votes
0answers
154 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
3
votes
0answers
35 views

Explicitly Show Conditional Independence In a Mixture Model

Suppose we have the following mixture model: $$\pi \sim Dirichlet(\alpha)$$ $$\theta_1 ,\ldots ,\theta_K \overset{iid}{\sim} N(0,1)$$ $$Z_1 , \ldots , Z_n \mid \pi \overset{iid}{\sim} Categorical(\...
3
votes
0answers
187 views

Dirichlet Process vs. Mixture Models with Many Mixtures

The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
3
votes
1answer
57 views

85% of the samples come from an unknown distribution, the rest come from the same distribution with a larger variance.How to recognize them?

Assume I have a data, say columns are the samples, and rows are the features, the problem is that around 85% of the samples come from an unknown distribution but the rest come from the same type of ...
3
votes
0answers
102 views

Feature Selection in Mixture of Experts Model

I am just beginning to learn about Mixture of Experts Models, so I apologize if parts of my question are elementary. My understanding is that a MEM is a mixture model where the components are linear ...
3
votes
0answers
88 views

Bayesian nonparametrics vs model selection using Minimum Message Length

As we know mixture models are important tools in density estimation and in general in statistical machine learning. I have always used nonparametric Bayesian mixture models to avoid the problem of ...
3
votes
0answers
207 views

Calculating conditional probability in Bernoulli mixture model

I'm following the book Pattern recognition and machine learning by Bishop on Bernoulli mixture model, and trying to code it. But I don't understand how to calculate the conditional probability (page ...
2
votes
0answers
19 views

Examples of mixture priors on SE-ARD length scale for Gaussian Process models?

Rasmussen and Williams (5.1) give the following notation for the SE-ARD kernel: $\begin{aligned}k(\mathbf{x}_p,\mathbf{x}_q)&=\sigma^2_f\hspace{0.5em} exp \left( -\frac{1}{2} (\mathbf{x}_p - \...
2
votes
0answers
360 views

Bayesian Nonparametric Latent feature model

For quite a long time I've been trying to understand the paper "Bayesian Nonparametric Latent feature model" (by Zoubin Ghahramani et al.) [http://mlg.eng.cam.ac.uk/zoubin/papers/GhaGriSol06.pdf]. In ...
2
votes
0answers
10 views

When can a finite mixture of distributions drawn from a distributional family be well-described by a distribution from the same family?

This question is motivated by a situation that we frequently encounter in economic variables. We believe that the overall distribution of some variable is well approximated by a particular ...
2
votes
0answers
34 views

What is the appropriate analysis for this type of repeated measures multi-binary data?

There is a popular theory within psychology that certain emotions will trigger "prototypical" facial expressions defined by the simultaneous contraction of specific facial muscles. For example, if a ...
2
votes
0answers
225 views

How to find the covariance of a Bernoulli Mixture distribution?

I am reading the Pattern Recognition and Machine Learning book and on page 445, it states that the covariance of the Bernoulli mixture distribution is $$Cov(\mathbf{x}) = \sum^K_{k=1} \pi_k (\Sigma_k ...
2
votes
0answers
684 views

Programming a mixture of a Gamma with a Normal distribution using R

I have some data x in R which seems to be a mixture of a Gamma and Normal distribution. Therefore I'd like to model this as a mixture model consisting of said distributions, but I don't know how to ...
2
votes
0answers
128 views

Does a latent class model become unstable with large samples and dimensions?

I have a dataset of approx 2million individuals with 40 dichotomous variables signifying presence or not of diseases (e.g heart disease, asthma, etc.) along with some sociodemographic info. I have ...
2
votes
0answers
886 views

fitting mixture of bimodal distribution

I am trying to fit my data with a bimodal distribution using two beta distributions, however it seems to me that the two peaks are not captured very well. The reason that I notice from the data is ...
2
votes
0answers
93 views

Statistical methods for adjusting heaping in birth weight data

I have a birth weight data of children under aged 5 years (sample size, n=2000). Data comes from two sources: Mothers recall (n=1500) and Health card (n=500). Naturally, mothers' recall data are ...
2
votes
0answers
498 views

Can I have a bimodal likelihood function to represent a mix of two populations?

I came across the following toy example and am lacking a final answer/step to finish the analysis. Imagine a surgery or medical procedure where we don't know the success-probability. It can be ...
1
vote
0answers
15 views

Choosing the Dirichlet prior in a mixture model

Consider the following mixture model with $K < \infty$ components, $$ f\left(x \mid \theta_{1}, \ldots, \theta_{K}, \pi_{1}, \ldots, \pi_{K}\right)=\sum_{k=1}^K \pi_{k} \varphi\left(x \mid \theta_{...
1
vote
0answers
16 views

How to fit a statistical distribution data to a mixure of normal and lognormal distributions

I have a data set that is a mixture of one normal and one lognormal distribution. How can I fit the data to get the fit parameters? Can somebody help me with this.
1
vote
0answers
8 views

Should cluster intercepts add up to the overall mean in latent class clustering?

I am working with flexmix package in R to cluster my data (1341 p, 8 observations each). My DV is scaled per respondent, but the overall mean still remains 0. I am using a latent class regression ...
1
vote
0answers
394 views

Generalised Linear Mixed Effect Models using latent class probabilites as weights

I'm relatively new to the field of generalised linear mixed models (GLMM) so this may be a redundant question but anyway... I'm trying to create a regression model to predict a binary outcome where I ...
1
vote
0answers
15 views

Conditional draws from a multivariate mixture model keeping one variable fixed

I would like to draw samples from a multivariate mixture model, for a given value of one of the variables. Assuming a gaussian mixture distribution built on $P_{1..p}$ variables with $K$ components: ...
1
vote
0answers
17 views

How to sample from a mixture of densities of transformed random variables?

Suppose we are given a set of $m$ random variables, $X_1$, $X_2$, ..., $X_m$, defined over the same set $\mathcal{X}$, with known densities $p_{X_i}$, for $i=1$, $2$, ..., $m$. Assume that getting ...
1
vote
1answer
420 views

Calculation of AIC in finite mixture modeling

I have a question about calculation the AIC to find my optimal amount of clusters. I am applying mixture modeling with the EM algorithm. I know the formula AIC = -2ln(log-lik) + 2k. These are my log-...
1
vote
0answers
35 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
1
vote
0answers
26 views

A 2 component mixture is symmetric if and only if $\lambda\in \{0,1,\frac{1}{2}\}$

Consider the following mixture of two densities $$ f(x)=\lambda g(x-\mu_1)+(1-\lambda)g(x-\mu_2) $$ with $\lambda\in [0,1]$, $g(\cdot)$ symmetric around zero, $\mu_1<\mu_2$. Claim: the mixture is ...
1
vote
0answers
28 views

Model Selection and inference for mixture of logisitc regressions (or GLM) with heterogenous covariates by component

I am facing a problem which should be quite common IMO but for which I don't find relevant contribution. So the situation is this. Let's say that a binary response $Y$ is generated by a mixture of $K$ ...
1
vote
0answers
521 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
1
vote
0answers
213 views

Inference with Mixture of Linear Regression

I have used an EM algorithm to fit a finite mixture of linear regression to my data, and cluster them into $k$ clusters. Now that I have my clusters with the estimated parameters $\beta_k$ and $\...
1
vote
0answers
40 views

calculate percent of sample in overlapping distributions

I'm looking at 80k samples representing results of a test after a fixed period of time. I think it looks like a combination of normal curves. I see a fair amount of documentation about combining ...
1
vote
0answers
154 views

Gaussian Mixture Modeling - Determining More Than One Component

Let us follow the convention that a lower information criteria score is considered better. Suppose we have a ground-truth Gaussian mixture model (GMM) with $k$ components. Suppose also that we (1) ...
0
votes
0answers
25 views

Testing a sample is drawn from a mixture

Suppose you have data $(Y_1, \dots, Y_N)$ drawn from a finite mixture population with $K$ components and you estimate the model parameters $(\theta_1,\dots, \theta_K)$ and the mixture weights $(\phi_1,...
0
votes
0answers
17 views

Multilevel 3-step latent class analysis: Accounting for group at multiple phases of the 3-step method?

I conducted a multilevel latent class analysis with a large dataset gathered from five different sites, with site specified as the grouping variable. The best fitting model included 2 classes and only ...
0
votes
0answers
5 views

Estimating the parameters for a true population when some of my data is spurious: can I use a mixture model?

I have a dataset where some of my data is "real" (i.e. of interest) and some is "spurious" (not of interest). I want to estimate some correlations between variables in the real ...
0
votes
0answers
6 views

Importance sampling from a mixture model

Let us suppose to have the following mixture model $j \sim Cat(j|\pi)$, $x \sim p(x|j)$. Suppose that I observe a dataset of $\{j_i, x_i, f_i\}_{i=1}^N$, where $j_i \sim Cat(j|\pi), x_i \sim p(x_i|J_i)...
0
votes
0answers
24 views

How to make inference on cluster-specific parameters in a Bayesian mixture model

Suppose I have a mixture model, for example of the kind $$ y_i \mid w, \{\theta_h\}, H \sim \sum_{h=1}^H w_h f(y_i \mid \theta_h) \\ P(H=h) = q_h \\ w \mid H \sim Dirichlet(\alpha) \\ \theta_1, \ldots,...
0
votes
0answers
43 views

Mixture model for decomposing bimodal or multimodal distributions

The gaussian mixture model (GMM) is fed mixture components or features whose time series each have differing means and variances from one another, but are unimodal (have one mode) with each component ...
0
votes
0answers
10 views

Finite mixture model, nonlinear, parametric, individuals

Suppose that I think individual $i$ chooses number $c$ according to the formula $c = v(f(\delta, \gamma) x - \alpha y))$ where $v$ and $f$ are nonlinear functions. I know what form $v$ and $f$ take. I ...
0
votes
0answers
86 views

Mixture model for a mix of normal and lognormal distributions in R

I have a distribution composed of a mixture of a normal distribution and a log-normal distribution. A simulated example that looks quite similar to what I will expect in the real data would be: ...
0
votes
0answers
8 views

Explaining Experimental data with EUT or PT: Using Structural Equation modeling or Fixed Mixture model?

I will try to be as clear as possible. Some background: I am doing experiments in economics and I have a huge dataset of experiments mirroring real life tax declaration. Participants in the lab ...
0
votes
0answers
31 views

Compute membership probabilities in E-step of EM algorithm with log-densities instead of densities

As an exercise I have implemented the EM algorithm for Gaussian mixtures, however, I have the problem that in high dimensions the densities of data points become so small that I get a numerical ...
0
votes
0answers
17 views

How do I correctly specify bayesian zero-inflated finite mixture model in jags?

I have basic understanding of Bayesian models and can fit simple models using "rjags". However, I am trying to fit a zero-inflated finite mixture model with n components (say n=2). I expect each ...
0
votes
2answers
229 views

EM algorithm for mixture of categorical distributions instantly stabilizes

Brief Summary of Question I'm trying to fit a mixture model of categorical distributions (see https://en.wikipedia.org/wiki/Categorical_distribution). The expectation at the second time step is ...
0
votes
0answers
11 views

How can I assess the number of cases needed to distinguish whether responses are from one of several probability distributions?

In my experiment, participants choose one of three options on every trial (the trials are 3x3 versions of the well known Prisoner's Dilemma game). What decision rule they use determines the ...
0
votes
0answers
10 views

Is this a typo in the `mclust` exposition?

I'm reading about the mclust package for gaussian mixture models. I want to understand its statistical assumptions. https://link.springer.com/content/pdf/10.1007/...
0
votes
0answers
405 views

Comparing 2 mixture models using mixtools

I have 2 mixture models I'd like to compare. Specifically, I want to compare lamda (i.e. proportion/area under each distribution) as it looks like there are differences there. Is this possible? ...
0
votes
1answer
33 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
0
votes
0answers
82 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...