Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

0
votes
1answer
10 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
1
vote
1answer
34 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
0
votes
1answer
21 views

Concise way to visualize / compare many Gaussian mixtures

I have 5,000 samples drawn from each of approximately 50,000 distributions. I have good reason to expect most of them to be normally distributed, and I expect some of them to be multi-modal (mixture ...
0
votes
0answers
28 views

Determine overlap of mixture models

I have a modeling problem where the time between events can be approximated as a mixture model of 2 Gaussians ($\mu=14, \sigma=5, \lambda=0.8$ and $\mu=6, \sigma=2,\lambda=0.2$). The response to ...
0
votes
0answers
8 views

Mixture Density Network: How to interpret the training behavior?

Mixture Density Neural Network: How to interpret the training behavior? I am trying to train a convolutional neural network that predicts the parameters of an output variable distribution using a 2D ...
0
votes
0answers
38 views

AIC /BIC for Gaussian Mixture models

Is it possible for BIC to keep decreasing with increasing number of clusters as shown in the figure. In this case, how do I choose the ideal number of components for Gaussian Mixture Models
0
votes
0answers
10 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
133 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
3
votes
1answer
70 views

How to cluster parts of broken line made of points?

I am studying clustering techniques and i am pretty new at this topic. Here is my problem: I created a 5 lines which are made of points. This lines are supposed to be continuous and they look like ...
3
votes
2answers
179 views

Gaussian mixture model - does an improper uniform prior give a proper posterior?

We draw $n$ i.i.d. points $x_1 , x_2 , ..., x_n$ from the following Gaussian mixture: $$p(x|\mu_1,\mu_2) = \frac{1}{2} \text{N} (x|\mu_1,1) + \frac{1}{2} \text{N} (x|\mu_2,1).$$ The prior is the ...
0
votes
1answer
33 views

Are Gaussian Mixture Models stochastic or deterministic?

Each time we generate a gmm model, we obtain slightly different clusters. Can we hence say gmm is stochastic? We obtain the same clusters if a random seed is set; does this mean given a random seed, ...
1
vote
0answers
9 views

Does mixture of sigmoids make sense given the theories about mixture of bernoullis?

Mixture of bernoullis is the combination of bernoulli distributions, which can be illustrated by the sampling process of K bags of D coins, here is a quick tutorial about it https://cedar.buffalo.edu/~...
1
vote
0answers
27 views

Determining number of components in mixtures of normal distributions with common mean

This is a pretty simple question, suppose we want to fit a mixture distribution of multivariate normals with common mean $$y_i \sim \sum_k \pi_k N(\mu, \Sigma_k)$$ What is the preferred approach for ...
1
vote
1answer
31 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
3
votes
0answers
21 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
0
votes
0answers
4 views

Guassian Mixture Classification: interpretating component x variable means matrix

The Guassian Mixture model output by mclust::Mclust() function has a $parameters$mean element which is a matrix with dimensions ...
1
vote
0answers
13 views

Is there any background for constraining covariances on fitting GMM?

On clustering data using GMM model, I often see the option to constrain covariances of each clustered GMM. For example, http://scikit-learn.org/0.16/auto_examples/mixture/plot_gmm_classifier.html ...
0
votes
0answers
32 views

Output Size of Mixture Density Networks

I am working on a neural network in which the final output layer will be a Mixture Density Network (MDN), but am confused about the shape of the values that final layer should return. In the paper in ...
2
votes
1answer
31 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
1
vote
2answers
129 views

Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?

I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like: Since this was ...
0
votes
0answers
20 views

Combining GMM's in different dimensions

I have input $X$, which follows a distribution $P(X)$, which is best modeled using a mixture of Gaussians. I also have another random variable $T$, which is also best modeled by a mixture of Gaussians....
0
votes
1answer
49 views

Is it important to make a feature scaling before using Gaussian Mixture Model?

Is it important to make a feature scaling before using Gaussian Mixture Model? and why is it important while we are using probability in getting our clusters's parameters (mean and covariance matrix). ...
1
vote
0answers
170 views

Gaussian Mixture Model with labels in Python

I have data X and corresponding labels y and want to fit a Gaussian Mixture model to it. In Matlab, one has the option of specifying initial labels. I am trying to do the same in Python. This is what ...
2
votes
1answer
47 views

Correlated random variables from mixture distributions

Let I have three random variables whose density is a mixture of two Normals with these parameters: $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$ $\...
4
votes
2answers
294 views

Is a GMM-HMM equivalent to a no-mixture HMM enriched with more states?

I'm trying to model sequence data that has 5 hidden states. Observation data conditional to each state is gaussian except for one state for which mixture of 2 gaussians seems more appropriate. ...
2
votes
1answer
443 views

A Gaussian Mixture Model Is a Universal Approximator of Densities

When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following: A Gaussian mixture model is a universal approximator of densities, in the ...
0
votes
1answer
37 views

Decompose/split a single multivariate gauss into random gaussian mixture

Say, there is a single $n$-dimensional multivariate Gaussian. $$Gauss_a(\mu_a,\Sigma_a) $$ $\mu_a$ is $1\times n$ vector and $\Sigma_a$ is $n\times n$ matrix. Is there any easy way to decompose/...
0
votes
1answer
32 views

Why do we assume Gaussian margins in Gaussian mixture models [duplicate]

A Gaussian mixture model is a weighted sum of Gaussian densities, i.e., $L(\theta) = \sum_{i=1}^{m} \pi_{i} f(x_i)$ where $m$ is the number of the mixture component. Hence, Gaussian mixture ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
1
vote
0answers
40 views

Conditional sampling from a multivariate Gaussian Mixture

I am using scikit-learn to fit a gaussian mixture on a non-parametric multivariate distribution with three variables $ \mathbf{X} = (X_1, X_2, X_3) $ I want to sample from that distribution given ...
1
vote
0answers
40 views

Entropy of a mixture of Gaussians

I need to estimate as fast and accurately as possible the differential entropy of a mixture of $K$ multivariate Gaussians: $$ \mathcal{H}[q] = -\sum_{k=1}^K w_k \int q_k(\textbf{x}) \log \left[\sum_{...
1
vote
0answers
28 views

Can the hidden states of a HMM be interpreted as number of clusters underlying the data?

Trying to understand the physical significance of the number of hidden states of a HMM. Should they be interpreted as number of clusters in the data? If not, why? Or they should be interpreted as the ...
1
vote
0answers
15 views

MIxture model in R to generate noise in data

I have a bit of code in R that adds noise to an harmonic series according to a normal distribution: ...
0
votes
1answer
79 views

Does assumption of normality of each mixture components implies that each margins is normal

I just would like to understand some information about the joint normality and the margins. I read that the normal joint distribution almost always implies that the univariate margins are all normal. ...
2
votes
1answer
50 views

Introduction to Gaussian mixture models

First of all, I am sorry if this question is not acceptable by some of the readers. However, I really read many, many sources about Gaussian mixture models, but all what I found was a short tutorial ...
0
votes
2answers
54 views

Does the distribution of each mixture components have any information of the margins of the variables? [closed]

I just start working with Gaussian mixture models and I just confused about some information, which I really would like to make sure that I understand the model very well. A Gaussian mixture model ...
2
votes
1answer
168 views

Gaussian Mixture Model

with the following code I fit a Gaussian Mixture Model to arbitrarily created data. The code is working. The only thing I encounter is that during the calculation of the multivariate_normal I ...
1
vote
0answers
26 views

Statistical test for comparing two-gaussian mixture

I have a distribution of shape sizes under two different (biological) conditions. From prior knowledge, I do expect there to be two populations. I fit each condition to a two-Gaussian mixture model. ...
1
vote
1answer
63 views

Gaussian mixture models with constrained mixing proportions

I am fitting a Gaussian mixture model to multivariate data and my application suggests constraining the mixing proportions to lie in a pre-determined sub-space. I am curious if such an approach has ...
0
votes
0answers
57 views

How to interpret a GMM and a predict_GMM output

I would like to do outlier detection in a dataset by using e Mixture Model. I'm using the R functions GMM and predict_GMM but I can't understand which is the outlier factor and how to indentify ...
1
vote
0answers
71 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
0
votes
0answers
179 views

Calculating BIC for sklearn.mixture BayesianGaussianMixture

The GaussianMixture function easily allows calculation of BIC. In the source code it is defined as: ...
0
votes
0answers
31 views

Clustering with mclust and cluster packages in R [duplicate]

I need to group my data into few buckets and wanted to use clustering with R. It is important that I cluster it right as my downstream processing largely depends on how well the data is clustered ...
0
votes
1answer
48 views

Gaussian Bayesian Networks and covariance calculation

I have difficulties in understanding the way of calculation of covariance matrix in Gaussian Bayesian Nets (from conditional to joint): The last formula is about to calculates covariance between ...
0
votes
0answers
35 views

How to select the best Baysian GMM with Dirichlet process

I am very new to GMM models and currently trying to make a unsupervised clustering on data. The data has 34 features (dimension). I am thinking of using GMM with Dirichlet process and trying to code ...
2
votes
1answer
149 views

Doesn't the non-Gaussian source assumption of ICA render it practically useless?

Gaussian distributions appear everywhere in nature, indeed this was largely the justification for most classical methods' reliance on assumption of normality. ICA assumes non-Gaussian sources, indeed ...
3
votes
0answers
20 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
2
votes
0answers
24 views

In a normal mixture model, what general happens if the number of groups to be summed far exceeds the dimensions of each normal?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_kI_d) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean vector $\mu$ and d-...
1
vote
0answers
303 views

Correct number of components in GMM according to BIC and AIC plots

I have applied GMM(Gaussian Mixture Model) to my data set and I have plotted the resulting BIC(Bayesian Information Criterion) and AIC(Akaike Information Criterion) for different number of components. ...
0
votes
1answer
198 views

General conditional distributions for multivariate Gaussian mixtures

My question is similar to this one but considers a more general situation. Suppose that $ \vec{x} = (x_1, \dots, x_d) $ and let $$ p(\vec{x}) = \sum_{k=1}^{n} \pi_k \mathcal{N}(\vec{x} | \mu_k, \...