Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

460 questions
Filter by
Sorted by
Tagged with
40k views

Clustering a dataset with both discrete and continuous variables

I have a dataset X which has 10 dimensions, 4 of which are discrete values. In fact, those 4 discrete variables are ordinal, i.e. a higher value implies a higher/better semantic. 2 of these discrete ...
18k views

If k-means clustering is a form of Gaussian mixture modeling, can it be used when the data are not normal?

I'm reading Bishop on EM algorithm for GMM and the relationship between GMM and k-means. In this book it says that k-means is a hard assign version of GMM. I'm wondering does that imply that if the ...
30k views

EM algorithm manually implemented

I want to implement the EM algorithm manually and then compare it to the results of the normalmixEM of mixtools package. Of ...
4k views

Why is optimizing a mixture of Gaussian directly computationally hard?

Consider the log likelihood of a mixture of Gaussians: $$l(S_n; \theta) = \sum^n_{t=1}\log f(x^{(t)}|\theta) = \sum^n_{t=1}\log\left\{\sum^k_{i=1}p_i f(x^{(t)}|\mu^{(i)}, \sigma^2_i)\right\}$$ I was ...
15k views

How to fit mixture model for clustering

I have two variables - X and Y and I need to make cluster maximum (and optimal) = 5. Let's ideal plot of variables is like following: I would like to make 5 clusters of this. Something like this: ...
1k views

Why Expectation Maximization is important for mixture models?

There are many literature emphasize Expectation Maximization method on mixture models (Mixture of Gaussian, Hidden Markov Model, etc.). Why EM is important? EM is just a way to do optimization and is ...
6k views

Singularity issues in Gaussian mixture model

In chapter 9 of the book Pattern recognition and machine learning, there is this part about Gaussian mixture model: To be honest I don't really understand why this would create a singularity. Can ...
295 views

References that justify use of Gaussian Mixtures

Gaussian mixture models (GMMs) are appealing because they are simple to work with both in analytically and in practice, and are capable of modeling some exotic distributions without too much ...
2k views

Relation between sum of Gaussian RVs and Gaussian Mixture

I know that a sum of Gaussians is Gaussian. So, how is a mixture of Gaussians different? I mean, a mixture of Gaussians is just a sum of Gaussians (where each Gaussian is multiplied by the respective ...
6k views

Quantiles from the combination of normal distributions

I have information on the distributions of anthropometric dimensions (like shoulder span) for children of different ages. For each age and dimension, I have mean, standard deviation. (I also have ...
8k views

Different covariance types for Gaussian Mixture Models

While trying Gaussian Mixture Models here, I found these 4 types of covariances. ...
6k views

Mclust model selection

The R package mclust uses BIC as a criteria for cluster model selection. From my understanding, a model with the lowest BIC should be selected over other models (if ...
5k views

2-Gaussian mixture model inference with MCMC and PyMC

The problem I want fit the model parameters of a simple 2-Gaussian mixture population. Given all the hype around Bayesian methods I want to understand if for this problem Bayesian inference is a ...
844 views

Is there a concept of “enough” data for training statistical models?

I work on quite a lot of statistical modelling, such as Hidden Markov Models and Gaussian Mixture Models. I see that training good models in each of these cases requires a large (> 20000 sentences for ...
3k views

Distance between two Gaussian mixtures to evaluate cluster solutions

I'm running a quick simulation to compare different clustering methods, and currently hit a snag trying to evaluate the cluster solutions. I know of various validation metrics (many found in cluster....
7k views

When to use Gaussian mixture model?

I am new to using GMMs. I was not able to find any appropriate help online. Could anyone please provide me right resource on "How to decide if using GMM fits to my problem?" or in case of ...
2k views

Applying stochastic variational inference to Bayesian Mixture of Gaussian

I am trying to implement Gaussian Mixture model with stochastic variational inference, following this paper. This is the pgm of Gaussian Mixture. According to the paper, the full algorithm of ...
631 views

Does EM algorithm consistently estimate the parameters in Gaussian Mixture model?

I am studying the Gaussian Mixture model and come up with this question myself. Suppose the underlying data is generated from a mixture of $K$ Gaussian distribution and each of them has a mean vector ...
3k views

What is “mixture” in a gaussian mixture model

We often study Gaussian Mixture model as a useful model in machine learning and its applications. What is the physical significance of this "Mixture"? Is it used because a Gaussian Mixture Model ...
1k views

Simulate from a truncated mixture normal distribution

I want to simulate a sample from a mixture normal distribution such that $$p\times\mathcal{N}(\mu_1,\sigma_1^2) + (1-p)\times\mathcal{N}(\mu_2,\sigma_2^2)$$ is restricted to the interval $[0,1]$ ...
2k views

Why only the mean value is used in (K-means) clustering method?

In clustering methods such as K-means, the euclidean distance is the metric to use. As a result, we only calculate the mean values within each cluster. And then adjustments are made on the elements ...
3k views

When to use LDA over GMM for clustering?

I have a dataset containing user activity with 168 dimensions, where I want to extract clusters using unsupervised learning. It is not obvious to me whether to use a topic modelling approach in Latent ...
3k views

K-means as a limit case of EM algorithm for Gaussian mixtures with covariances $\epsilon^2 I$ going to $0$

My goal is to see that K-means algorithm is in fact Expectation-Maximization algorithm for Gaussian mixtures in which all components have covariance $\sigma^2 I$ in the limit as $\lim_{\sigma \to 0}$. ...
4k views

Fitting a gaussian mixture model using stochastic gradient descent

I'm working on an online category learning model which uses stochastic gradient descent to fit a gaussian mixture model. The model is based on the online learning model used in Toscano & McMurray (...
422 views

How to prove this Gaussian Mixture inequality? (Fitting/Overfitting)

Let f[x] be a Gaussian Mixture pdf with n terms of uniform weight, means $\{\mu_{1},...,\mu_{n}\}$, and corresponding variances $\{\sigma_{1},...,\sigma_{n}\}$: f(x)\equiv\frac{1}{n}\sum_{i=1}^{n}\...
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
276 views

Growing number of Gaussians in a mixture

Let I have a Gaussian mixture consisting of $n$ Gaussians that is already fitted (e.g. using EM algorithm) with respect to a given data set. Now I want to add one more Gaussian to make the mixture ...
992 views

Why use a Gaussian mixture model?

I am learning about Gaussian mixture models (GMM) but I am confused as to why anyone should ever use this algorithm. How is this algorithm better than other standard clustering algorithm such as $K$-...
853 views

Show that a scale mixtures of normals is a power exponential

I'm trying to show that a scale mixture of normals yields a Laplace distribution. I've gotten to the point where I have $\int N(0,\tau)\times Ga(\tau\:;\:1,\frac{\lambda^{2}}{2}) \:d\tau$ should equal ...
12k views

How to train a Gaussian mixture hidden Markov model?

I want to build a hidden Markov model (HMM) with continuous observations modeled as Gaussian mixtures (Gaussian mixture model = GMM). The way I understand the training process is that it should be ...
1k views

Finding the point of maximum probability in a mixture of gaussians

I have a model that estimates probability of an object to be located in a 2d space. Using a mixture of gaussian with a set of criteria that I chose I got interesting results, and now I am faced to a ...
1k views

Gaussian Mixture and Method of Moments

Given solely the first $n$ moments $m_1,\dots,m_n$ of a random variables $X\in\mathbb{R}$, I was wondering whether there exists a direct methodology to approximate $X$ with a Gaussian Mixture ?
629 views

choosing prior parameters for variational mixture of Gaussians

I am implementing a vanilla variational mixture of multivariate Gaussians, as per Chapter 10 of Pattern Recognition and Machine Learning (Bishop, 2007). The Bayesian approach requires to specify (...
2k views

Variance of EM mean estimates in a simple mixture of two normals

Consider a mixture of two normal distributions: $f(x) = p N(x|u_1, S_1) + (1-p) N(x|u_2, S_2)$ where N() is the normal pdf. $p$, $S_2$, and $S_2$ are known. The means are not. You can get the MLE ...
1k views

Why use a mixture model with RNN instead of just directly predictive real values?

Alex Graves created a model to generate hand writing sequences which use an LSTM (kind of Recurrent Neural Network) to predict the parameters for an mixture model. The mixture model is then used to ...
1k views

Gaussian mixture regression in higher dimensions

Problem: I have a discrete representation of a surface/height-map $z = f(x,y)$ that i want to model as a mixture of Gaussians (please take probability distributions out of your mind for a moment). ...
1k views

Skewness of a mixture density

I am fitting a gaussian mixture to financial data. My mixture density is given by: $f(l)=πϕ(l;μ_1,σ^2_1)+(1−π)ϕ(l;μ_2,σ_2^2)$ I calculated the skewness of the data already. Now, I want to look at ...
2k views

Gaussian mixture vs. Gaussian process

As far as I know, both Gaussian mixtures as well as Gaussian processes can be used for regression. My question is: what is better and why? The answers might contain theoretic insights, practical ...
127 views

844 views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
795 views

Why do we use Gaussian distributions in Variational Autoencoder?

I still don't understand why we force the distribution of the hidden representation of a Variational Autoencoder (VAE) to follow a multivariate normal distribution. Why this specific distribution and ...
541 views

How to tell if a mixture of Gaussians will be multimodal

Suppose I have a mixture of Gaussians and I know the mean and variance of each separate Gaussian. How can I tell whether or not the resulting distribution will be multimodal or, more specifically, ...
5k views

Gaussian Mixture Models: Maximum Likelihood Estimation or Expectation Maximization?

As far as I know the usual method for estimating the parameters in GMM is EM. However, it is also possible to use maximum likelihood. What are the differences between these two methods? Why would one ...
520 views

Choosing a model for my unsupervised machine learning problem

I need to choose a model for unsupervised machine learning problem. There are 4 clusters in 3D space. These are my requirements: I will run the same model multiple times with different training data (...
2k views

Sum of Gaussian mixture and Gaussian scale mixture

What will be the distribution of the sum of two independent random variables, say $X$ and $Y$, when $X$ has a Gaussian mixture distribution (when we take Gaussian distribution with different location ...
Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is unknown....