Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

0
votes
1answer
15 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
0
votes
0answers
8 views

Identification of mixture models with location shift

I am studying identification of mixture models of the type $$ F(x)=\sum_{j=1}^J \lambda_j G(x-\mu_j) $$ with $F,G$ denoting univariate CDFs, $G$ symmetric about $0$, $\mu_1<\mu_2<...<\mu_J$, $...
1
vote
1answer
27 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
3
votes
1answer
22 views

Is it acceptable in mathematical saying that the E-step is equal to posterior probability

I am studying EM-algorithm for mixture data. I read that some authors said that, the E-step is equivalent to the calculation of the posterior probability (I think this come from Bayesian rule). So, my ...
0
votes
1answer
19 views

Equivalent way of rewriting a two-component mixture

I'm confused on the following equivalent way of rewriting a two-component mixture. Consider the two-component conditional mixture $$ F(z|x)=\lambda F_1(z|x)+(1-\lambda)F_2(z|x) $$ where all the $F$'...
1
vote
0answers
25 views

A 2 component mixture is symmetric if and only if $\lambda\in \{0,1,\frac{1}{2}\}$

Consider the following mixture of two densities $$ f(x)=\lambda g(x-\mu_1)+(1-\lambda)g(x-\mu_2) $$ with $\lambda\in [0,1]$, $g(\cdot)$ symmetric around zero, $\mu_1<\mu_2$. Claim: the mixture is ...
0
votes
0answers
14 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
0
votes
0answers
22 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
0
votes
1answer
39 views

Mixture of two components

Consider a univariate probability density function $p(x)$ that is a mixture of $2$ probability density functions with weights $\eta, 1-\eta$ and $\eta\in (0,1)$: $$ p(x)=(1-\eta)g(x)+\eta f(x) \...
2
votes
1answer
34 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
169 views

Posterior distribution of mixture models

In the context of mixture models in Bayesian inference, one can assume that the general form of the joint posterior for a mixture model of $k$ components is $$ \begin{equation} p( \boldsymbol{\...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
3
votes
2answers
47 views

Confusion in modelling finite mixture model

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated: The usual representation (of a finite ...
3
votes
1answer
54 views

Generate a finite mixture distribution by sampling mixture component parameters

Assume I have a some mixture distribution, $H$, with mean $\mu$ and variance $\sigma^2$. $H$ is a mixture of $n$ component distributions where all component weights are equal. Let $\mu_i$ be the mean ...
0
votes
0answers
13 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
56 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
0
votes
1answer
122 views

PyMC3: Mixture Model with Latent Variables

I have a rather basic knowledge of Bayesian inference and I'm somewhat new to MCMC and PyMC3. Can I model data that looks like this? ...
2
votes
1answer
40 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
0
votes
0answers
77 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...
1
vote
0answers
24 views

Model Selection and inference for mixture of logisitc regressions (or GLM) with heterogenous covariates by component

I am facing a problem which should be quite common IMO but for which I don't find relevant contribution. So the situation is this. Let's say that a binary response $Y$ is generated by a mixture of $K$ ...
4
votes
1answer
58 views

Marginal Distribution of Exponential Mixture Model

I am currently trying to marginalize over the scale parameter in a mixture distribution of exponential pdfs, but I do not trust my result. Let me show you my steps: Probability Density Function The ...
2
votes
2answers
299 views

Question about the latent variable in EM algorithm

In mixture models, Expectation maximization algorithm (EM) is a commonly used method to estimate the model parameters. Suppose that I have bivariate mixture model with two mixture components, with ...
1
vote
0answers
21 views

What is the appropriate analysis for this type of repeated measures multi-binary data?

There is a popular theory within psychology that certain emotions will trigger "prototypical" facial expressions defined by the simultaneous contraction of specific facial muscles. For example, if a ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
-1
votes
2answers
38 views

Variance of a mixture of Normals with same $\sigma^2_i$

Let $Y\sim \sum^N_{i=1}\omega_iN(m_i,h^2 V)$. The text I'm reading states that $Var(Y)=(1+h^2)V$, when $m_i=\theta_i$, where $\theta_i$ are draws taken from $P(\theta|D)$, and $V=Var(\theta|D)$ I ...
1
vote
0answers
169 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
0
votes
0answers
19 views

simulating finite gaussian mixture models

I'm doing the project for my data science exam, I'm not actually a statistician so I had some doubts. This is my data set:y=c(9.259778,9.178891,9.262575,9.212859,9.241963,9.290917,9.178455,9.183814,9....
3
votes
0answers
24 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
2
votes
1answer
44 views

Fitting a finite mixture: choice of the distribution and model selection for the number of components

This is a question about finite mixture models (FMM). We want to fit a dataset $D$ but we are not 100% sure of which distributions we should use to create the mixture; we do not how many clusters ...
1
vote
1answer
169 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
0
votes
1answer
17 views

Why does the mixture dependencies exist between variables?

Sorry if my question is clear to most of you. As a mathematics background, I really just start working with the mixture and would like to understand it in a clear way. Mixture dependencies are when ...
0
votes
0answers
12 views

Can describe a dependencies as weak dependency?

Mixture dependencies structure can be very strong. For example, 60% of the data come from the first mixture component. However, sometime we may have a weak mixture ...
0
votes
0answers
18 views

LRT for betabinomial mixture model?

Is the LRT appropriate to compare two mixtures of betabinomials (or gaussians), one with k components, the other with k+1? I've read in some place that it is appropriate, but after that found papers ...
0
votes
1answer
125 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
0
votes
1answer
30 views

Very steep decrease in information criteria for mixture models with more components

I am analyzing data using mixture modeling. When I plot the information criteria (the BIC) for a series of models (with different model specifications and different number of mixture components), I ...
3
votes
1answer
626 views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
0
votes
0answers
145 views

Mixture modelling with skewed distributions

I am trying to find a R library that splits a distribution into a symmetric and asymmetric components. I have a distribution that I want to split into two components, one skewed and the other most ...
3
votes
0answers
26 views

Explicitly Show Conditional Independence In a Mixture Model

Suppose we have the following mixture model: $$\pi \sim Dirichlet(\alpha)$$ $$\theta_1 ,\ldots ,\theta_K \overset{iid}{\sim} N(0,1)$$ $$Z_1 , \ldots , Z_n \mid \pi \overset{iid}{\sim} Categorical(\...
0
votes
1answer
43 views

Why the sample method of mixture distribution works?

For example this thread: Generating random variables from a mixture of Normal distributions First choose a distribution according to the weights. Then sample from the chosen distribution. How to ...
6
votes
1answer
121 views

Observations for a bivariate Gaussian mixture

Consider two random vectors $X\equiv(X_1, X_2),Y\equiv(Y_1, Y_2)$ distributed as below 1) $X\sim N(\begin{pmatrix} \mu_{X,1}\\ \mu_{X,2}\\ \end{pmatrix}, \begin{pmatrix} v_{X,1} & 0\\ 0 & v_{...
0
votes
2answers
1k views

Mixture models vs Mixed models

I was wondering what is the difference between Mixture models and Mixed models in Statistics? Explaining with Any practical example would be appreciated.
3
votes
0answers
120 views

Dirichlet Process vs. Mixture Models with Many Mixtures

The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
0
votes
0answers
30 views

DPMM asymptotics for finite mixtures?

My understanding is that using a Dirichlet process mixture model for a mixture with finitely many components will result in a misspecified model. Are there any asymptotic results or bounds on the ...
3
votes
1answer
51 views

85% of the samples come from an unknown distribution, the rest come from the same distribution with a larger variance.How to recognize them?

Assume I have a data, say columns are the samples, and rows are the features, the problem is that around 85% of the samples come from an unknown distribution but the rest come from the same type of ...
2
votes
1answer
161 views

Does the sum of updated mixture weights in EM algorithm equal 1 in M step, why?

I am usign EM algorithm to estimate my model's parameters. As you know, the mixture weight must be sum to one. $\sum \pi_n = 1$ where $n$ is the number of mixture component. In M step we can find the ...
0
votes
1answer
205 views

Does a mixture model need to sum or integrate to $1$?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean $\mu$ and variance $\...
1
vote
1answer
137 views

What is the relationship between gaussian mixture models and covariate regression?

Consider a histogram of observations $y$ which look a bit like this, Imagine that the two bell-curve shaped histograms (ignore the actual fitted curves) correspond to observations from females and ...
1
vote
0answers
171 views

Inference with Mixture of Linear Regression

I have used an EM algorithm to fit a finite mixture of linear regression to my data, and cluster them into $k$ clusters. Now that I have my clusters with the estimated parameters $\beta_k$ and $\...
2
votes
0answers
149 views

How to find the covariance of a Bernoulli Mixture distribution?

I am reading the Pattern Recognition and Machine Learning book and on page 445, it states that the covariance of the Bernoulli mixture distribution is $$Cov(\mathbf{x}) = \sum^K_{k=1} \pi_k (\Sigma_k ...
1
vote
0answers
31 views

calculate percent of sample in overlapping distributions

I'm looking at 80k samples representing results of a test after a fixed period of time. I think it looks like a combination of normal curves. I see a fair amount of documentation about combining ...