# Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

548 questions
Filter by
Sorted by
Tagged with
49k views

### Clustering a dataset with both discrete and continuous variables

I have a dataset X which has 10 dimensions, 4 of which are discrete values. In fact, those 4 discrete variables are ordinal, i.e. a higher value implies a higher/better semantic. 2 of these discrete ...
22k views

### If k-means clustering is a form of Gaussian mixture modeling, can it be used when the data are not normal?

I'm reading Bishop on EM algorithm for GMM and the relationship between GMM and k-means. In this book it says that k-means is a hard assign version of GMM. I'm wondering does that imply that if the ...
17k views

### Different covariance types for Gaussian Mixture Models

While trying Gaussian Mixture Models here, I found these 4 types of covariances. ...
34k views

### EM algorithm manually implemented

I want to implement the EM algorithm manually and then compare it to the results of the normalmixEM of mixtools package. Of ...
5k views

### Why is optimizing a mixture of Gaussian directly computationally hard?

Consider the log likelihood of a mixture of Gaussians: $$l(S_n; \theta) = \sum^n_{t=1}\log f(x^{(t)}|\theta) = \sum^n_{t=1}\log\left\{\sum^k_{i=1}p_i f(x^{(t)}|\mu^{(i)}, \sigma^2_i)\right\}$$ I was ...
2k views

### Why Expectation Maximization is important for mixture models?

There are many literature emphasize Expectation Maximization method on mixture models (Mixture of Gaussian, Hidden Markov Model, etc.). Why EM is important? EM is just a way to do optimization and is ...
11k views

### Singularity issues in Gaussian mixture model

In chapter 9 of the book Pattern recognition and machine learning, there is this part about Gaussian mixture model: To be honest I don't really understand why this would create a singularity. Can ...
7k views

### Quantiles from the combination of normal distributions

I have information on the distributions of anthropometric dimensions (like shoulder span) for children of different ages. For each age and dimension, I have mean, standard deviation. (I also have ...
16k views

### How to fit mixture model for clustering

I have two variables - X and Y and I need to make cluster maximum (and optimal) = 5. Let's ideal plot of variables is like following: I would like to make 5 clusters of this. Something like this: ...
385 views

### References that justify use of Gaussian Mixtures

Gaussian mixture models (GMMs) are appealing because they are simple to work with both in analytically and in practice, and are capable of modeling some exotic distributions without too much ...
3k views

### Relation between sum of Gaussian RVs and Gaussian Mixture

I know that a sum of Gaussians is Gaussian. So, how is a mixture of Gaussians different? I mean, a mixture of Gaussians is just a sum of Gaussians (where each Gaussian is multiplied by the respective ...
4k views

### Distance between two Gaussian mixtures to evaluate cluster solutions

I'm running a quick simulation to compare different clustering methods, and currently hit a snag trying to evaluate the cluster solutions. I know of various validation metrics (many found in cluster....
8k views

### Mclust model selection

The R package mclust uses BIC as a criteria for cluster model selection. From my understanding, a model with the lowest BIC should be selected over other models (if ...
3k views

### In cluster analysis, how does Gaussian mixture model differ from K Means when we know the clusters are spherical?

I understand how main difference between K-mean and Gaussian mixture model (GMM) is that K-Mean only detects spherical clusters and GMM can adjust its self to elliptic shape cluster. However, how do ...
5k views

### Why do we use Gaussian distributions in Variational Autoencoder?

I still don't understand why we force the distribution of the hidden representation of a Variational Autoencoder (VAE) to follow a multivariate normal distribution. Why this specific distribution and ...
2k views

### Simulate from a truncated mixture normal distribution

I want to simulate a sample from a mixture normal distribution such that $$p\times\mathcal{N}(\mu_1,\sigma_1^2) + (1-p)\times\mathcal{N}(\mu_2,\sigma_2^2)$$ is restricted to the interval $[0,1]$ ...
6k views

### 2-Gaussian mixture model inference with MCMC and PyMC

The problem I want fit the model parameters of a simple 2-Gaussian mixture population. Given all the hype around Bayesian methods I want to understand if for this problem Bayesian inference is a ...
910 views

### Is there a concept of “enough” data for training statistical models?

I work on quite a lot of statistical modelling, such as Hidden Markov Models and Gaussian Mixture Models. I see that training good models in each of these cases requires a large (> 20000 sentences for ...
2k views

### Why use a Gaussian mixture model?

I am learning about Gaussian mixture models (GMM) but I am confused as to why anyone should ever use this algorithm. How is this algorithm better than other standard clustering algorithm such as $K$-...
2k views

### Applying stochastic variational inference to Bayesian Mixture of Gaussian

I am trying to implement Gaussian Mixture model with stochastic variational inference, following this paper. This is the pgm of Gaussian Mixture. According to the paper, the full algorithm of ...
4k views

### When to use LDA over GMM for clustering?

I have a dataset containing user activity with 168 dimensions, where I want to extract clusters using unsupervised learning. It is not obvious to me whether to use a topic modelling approach in Latent ...
10k views

### When to use Gaussian mixture model?

I am new to using GMMs. I was not able to find any appropriate help online. Could anyone please provide me right resource on "How to decide if using GMM fits to my problem?" or in case of ...
6k views

### Fitting a gaussian mixture model using stochastic gradient descent

I'm working on an online category learning model which uses stochastic gradient descent to fit a gaussian mixture model. The model is based on the online learning model used in Toscano & McMurray (...
1k views

### choosing prior parameters for variational mixture of Gaussians

I am implementing a vanilla variational mixture of multivariate Gaussians, as per Chapter 10 of Pattern Recognition and Machine Learning (Bishop, 2007). The Bayesian approach requires to specify (...
961 views

### Does EM algorithm consistently estimate the parameters in Gaussian Mixture model?

I am studying the Gaussian Mixture model and come up with this question myself. Suppose the underlying data is generated from a mixture of $K$ Gaussian distribution and each of them has a mean vector ...
1k views

### Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
4k views

### What is “mixture” in a gaussian mixture model

We often study Gaussian Mixture model as a useful model in machine learning and its applications. What is the physical significance of this "Mixture"? Is it used because a Gaussian Mixture Model ...
2k views

### Finding the point of maximum probability in a mixture of gaussians

I have a model that estimates probability of an object to be located in a 2d space. Using a mixture of gaussian with a set of criteria that I chose I got interesting results, and now I am faced to a ...
3k views

### Why only the mean value is used in (K-means) clustering method?

In clustering methods such as K-means, the euclidean distance is the metric to use. As a result, we only calculate the mean values within each cluster. And then adjustments are made on the elements ...
2k views

### Difference between GMM classification and QDA

I know that every class has the same covariance matrix $\Sigma$ in linear discriminant analysis (LDA), and in quadratic discriminant analysis (QDA) they are different. When using gaussian mixture ...
2k views

### Gaussian Mixture and Method of Moments

Given solely the first $n$ moments $m_1,\dots,m_n$ of a random variables $X\in\mathbb{R}$, I was wondering whether there exists a direct methodology to approximate $X$ with a Gaussian Mixture ?
4k views

### K-means as a limit case of EM algorithm for Gaussian mixtures with covariances $\epsilon^2 I$ going to $0$

My goal is to see that K-means algorithm is in fact Expectation-Maximization algorithm for Gaussian mixtures in which all components have covariance $\sigma^2 I$ in the limit as $\lim_{\sigma \to 0}$. ...
436 views

### How to prove this Gaussian Mixture inequality? (Fitting/Overfitting)

Let f[x] be a Gaussian Mixture pdf with n terms of uniform weight, means $\{\mu_{1},...,\mu_{n}\}$, and corresponding variances $\{\sigma_{1},...,\sigma_{n}\}$: f(x)\equiv\frac{1}{n}\sum_{i=1}^{n}\...
2k views

### Gaussian mixture regression in higher dimensions

Problem: I have a discrete representation of a surface/height-map $z = f(x,y)$ that i want to model as a mixture of Gaussians (please take probability distributions out of your mind for a moment). ...
298 views

### Growing number of Gaussians in a mixture

Let I have a Gaussian mixture consisting of $n$ Gaussians that is already fitted (e.g. using EM algorithm) with respect to a given data set. Now I want to add one more Gaussian to make the mixture ...
7k views

876 views

### Fitting Gaussian mixture models with dirac delta functions

I was told that using gradient methods for Gaussian mixture models may end up with Dirac delta function(s). I hadn't thought of this problem before, but when I verify this, it does seem to be a ...
2k views

### Variance of EM mean estimates in a simple mixture of two normals

Consider a mixture of two normal distributions: $f(x) = p N(x|u_1, S_1) + (1-p) N(x|u_2, S_2)$ where N() is the normal pdf. $p$, $S_2$, and $S_2$ are known. The means are not. You can get the MLE ...
1k views

### Why use a mixture model with RNN instead of just directly predictive real values?

Alex Graves created a model to generate hand writing sequences which use an LSTM (kind of Recurrent Neural Network) to predict the parameters for an mixture model. The mixture model is then used to ...
6k views

### Gaussian Mixture Models: Maximum Likelihood Estimation or Expectation Maximization?

As far as I know the usual method for estimating the parameters in GMM is EM. However, it is also possible to use maximum likelihood. What are the differences between these two methods? Why would one ...
725 views

### How to tell if a mixture of Gaussians will be multimodal

Suppose I have a mixture of Gaussians and I know the mean and variance of each separate Gaussian. How can I tell whether or not the resulting distribution will be multimodal or, more specifically, ...
2k views

### Skewness of a mixture density

I am fitting a gaussian mixture to financial data. My mixture density is given by: $f(l)=πϕ(l;μ_1,σ^2_1)+(1−π)ϕ(l;μ_2,σ_2^2)$ I calculated the skewness of the data already. Now, I want to look at ...
1k views

### Why are hidden Markov models (HMM) also called mixture models?

Why are hidden Markov models (HMM) called mixture models? What does it mix?
1k views

### Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
When implementing GMM (Gaussian Mixture Model) in practice, the covariance matrix ${\Sigma}_{D\times D}$ is often singular. The reason is that we have to estimate $\frac{D(D+1)}{2}$ parameters in \$\...