Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

Filter by
Sorted by
Tagged with
1
vote
0answers
12 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
4
votes
1answer
70 views

How to latently cluster regressors based on regressors' relationship with the outcome?

What is the best way/method to model patterns across coefficients and reduce number of coefficients based on these patterns? We have hundreds of regressors on the same scale and try to reduce the ...
0
votes
1answer
18 views

Is there a way to determine the number of the mixture components prior to run EM algorithm

I am working with mixture models. The common way to determine the number of the mixture components is fitting several mixture models with a different number of mixture components and then select the ...
0
votes
1answer
20 views

Estimating two-component mixture of Weibull distributions?

Is there any existing package (preferably in Python or Matlab) to estimate the parameters of a two-component Weibull mixture model? And failing that, I am hoping to get some pointers towards rolling ...
0
votes
0answers
12 views

Data distributions extrapolation

Is there a way to extrapolate (according to variance or other statistics) distributions from data? In other words, I have a gene expression matrix in which rows are genes while columns are samples (i....
0
votes
0answers
26 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
1
vote
1answer
29 views

Latent profile analysis - Gaussian models and which one to choose?

When performing LPA different models are possible depending on shape, volume and orientation (Figure 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5096736/). Can someone, please, explain in simple ...
2
votes
0answers
7 views

When can a finite mixture of distributions drawn from a distributional family be well-described by a distribution from the same family?

This question is motivated by a situation that we frequently encounter in economic variables. We believe that the overall distribution of some variable is well approximated by a particular ...
0
votes
0answers
17 views

Piecewise Pareto Distribution

What are the best practices for Piece-wise Pareto Distribution or maybe Pareto Mixture Model(?). Example: $x\in [0, 1) \Rightarrow \alpha=0.1$ $x\in [1, 10) \Rightarrow \alpha=0.5$ $x\in [10, 100) ...
0
votes
0answers
43 views

finite mixture models with countreg and flexmix

I am testing making a finite mixture model for poisson and negative binomial on some artificial data using flexmix and countreg packages. Here is the core part of my small script: ...
0
votes
0answers
10 views

Joint “density” of data and indicators in Bayesian mixture model

I'm currently working through the chapter on finite mixture models in BDA3 and came across the following model setup (with the usual slight abuse of notation): Let $\lambda=(\lambda_1,\dots,\lambda_H)...
0
votes
0answers
27 views

Seeking guidance on determining (predicting) probability (distribution) using Mixture Density (Neural) Networks

I work for an education PAAS (platform as a service) company helping colleges & training institutes manage their course evaluation and recruitment. Students can re-take tests multiple times, tests ...
0
votes
0answers
29 views

Using mixture models as a prediction model

I read many papers that used mixture model to predict, for example, The diagnosis of a disease. For example, assume that there are a measurements on different variables for healthy patient and ...
0
votes
1answer
18 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
0
votes
0answers
9 views

Identification of mixture models with location shift

I am studying identification of mixture models of the type $$ F(x)=\sum_{j=1}^J \lambda_j G(x-\mu_j) $$ with $F,G$ denoting univariate CDFs, $G$ symmetric about $0$, $\mu_1<\mu_2<...<\mu_J$, $...
2
votes
1answer
28 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
3
votes
1answer
26 views

Is it acceptable in mathematical saying that the E-step is equal to posterior probability

I am studying EM-algorithm for mixture data. I read that some authors said that, the E-step is equivalent to the calculation of the posterior probability (I think this come from Bayesian rule). So, my ...
0
votes
1answer
20 views

Equivalent way of rewriting a two-component mixture

I'm confused on the following equivalent way of rewriting a two-component mixture. Consider the two-component conditional mixture $$ F(z|x)=\lambda F_1(z|x)+(1-\lambda)F_2(z|x) $$ where all the $F$'...
1
vote
0answers
26 views

A 2 component mixture is symmetric if and only if $\lambda\in \{0,1,\frac{1}{2}\}$

Consider the following mixture of two densities $$ f(x)=\lambda g(x-\mu_1)+(1-\lambda)g(x-\mu_2) $$ with $\lambda\in [0,1]$, $g(\cdot)$ symmetric around zero, $\mu_1<\mu_2$. Claim: the mixture is ...
0
votes
0answers
15 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
0
votes
0answers
26 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
0
votes
1answer
41 views

Mixture of two components

Consider a univariate probability density function $p(x)$ that is a mixture of $2$ probability density functions with weights $\eta, 1-\eta$ and $\eta\in (0,1)$: $$ p(x)=(1-\eta)g(x)+\eta f(x) \...
2
votes
1answer
51 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
220 views

Posterior distribution of mixture models

In the context of mixture models in Bayesian inference, one can assume that the general form of the joint posterior for a mixture model of $k$ components is $$ \begin{equation} p( \boldsymbol{\...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
3
votes
2answers
54 views

Confusion in modelling finite mixture model

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated: The usual representation (of a finite ...
3
votes
1answer
69 views

Generate a finite mixture distribution by sampling mixture component parameters

Assume I have a some mixture distribution, $H$, with mean $\mu$ and variance $\sigma^2$. $H$ is a mixture of $n$ component distributions where all component weights are equal. Let $\mu_i$ be the mean ...
0
votes
0answers
24 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
92 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
0
votes
1answer
234 views

PyMC3: Mixture Model with Latent Variables

I have a rather basic knowledge of Bayesian inference and I'm somewhat new to MCMC and PyMC3. Can I model data that looks like this? ...
2
votes
1answer
48 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
0
votes
0answers
189 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...
1
vote
0answers
26 views

Model Selection and inference for mixture of logisitc regressions (or GLM) with heterogenous covariates by component

I am facing a problem which should be quite common IMO but for which I don't find relevant contribution. So the situation is this. Let's say that a binary response $Y$ is generated by a mixture of $K$ ...
4
votes
1answer
67 views

Marginal Distribution of Exponential Mixture Model

I am currently trying to marginalize over the scale parameter in a mixture distribution of exponential pdfs, but I do not trust my result. Let me show you my steps: Probability Density Function The ...
3
votes
2answers
460 views

Question about the latent variable in EM algorithm

In mixture models, Expectation maximization algorithm (EM) is a commonly used method to estimate the model parameters. Suppose that I have bivariate mixture model with two mixture components, with ...
1
vote
0answers
23 views

What is the appropriate analysis for this type of repeated measures multi-binary data?

There is a popular theory within psychology that certain emotions will trigger "prototypical" facial expressions defined by the simultaneous contraction of specific facial muscles. For example, if a ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
-1
votes
2answers
44 views

Variance of a mixture of Normals with same $\sigma^2_i$

Let $Y\sim \sum^N_{i=1}\omega_iN(m_i,h^2 V)$. The text I'm reading states that $Var(Y)=(1+h^2)V$, when $m_i=\theta_i$, where $\theta_i$ are draws taken from $P(\theta|D)$, and $V=Var(\theta|D)$ I ...
1
vote
0answers
232 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
3
votes
0answers
34 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
2
votes
1answer
56 views

Fitting a finite mixture: choice of the distribution and model selection for the number of components

This is a question about finite mixture models (FMM). We want to fit a dataset $D$ but we are not 100% sure of which distributions we should use to create the mixture; we do not how many clusters ...
1
vote
1answer
217 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
0
votes
1answer
17 views

Why does the mixture dependencies exist between variables?

Sorry if my question is clear to most of you. As a mathematics background, I really just start working with the mixture and would like to understand it in a clear way. Mixture dependencies are when ...
0
votes
1answer
215 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
0
votes
1answer
30 views

Very steep decrease in information criteria for mixture models with more components

I am analyzing data using mixture modeling. When I plot the information criteria (the BIC) for a series of models (with different model specifications and different number of mixture components), I ...
3
votes
1answer
762 views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
0
votes
0answers
200 views

Mixture modelling with skewed distributions

I am trying to find a R library that splits a distribution into a symmetric and asymmetric components. I have a distribution that I want to split into two components, one skewed and the other most ...
3
votes
0answers
27 views

Explicitly Show Conditional Independence In a Mixture Model

Suppose we have the following mixture model: $$\pi \sim Dirichlet(\alpha)$$ $$\theta_1 ,\ldots ,\theta_K \overset{iid}{\sim} N(0,1)$$ $$Z_1 , \ldots , Z_n \mid \pi \overset{iid}{\sim} Categorical(\...
0
votes
1answer
88 views

Why the sample method of mixture distribution works?

For example this thread: Generating random variables from a mixture of Normal distributions First choose a distribution according to the weights. Then sample from the chosen distribution. How to ...
6
votes
1answer
126 views

Observations for a bivariate Gaussian mixture

Consider two random vectors $X\equiv(X_1, X_2),Y\equiv(Y_1, Y_2)$ distributed as below 1) $X\sim N(\begin{pmatrix} \mu_{X,1}\\ \mu_{X,2}\\ \end{pmatrix}, \begin{pmatrix} v_{X,1} & 0\\ 0 & v_{...