Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

0
votes
0answers
11 views

Dirichlet Process Mixture Conditioning

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
3
votes
2answers
59 views

Why do we use Gaussian distributions in Variational Autoencoder?

I still don't understand why we force the distribution of the hidden representation of a Variational Autoencoder (VAE) to follow a multivariate normal distribution. Why this specific distribution and ...
1
vote
0answers
32 views

Hidden Markov Model for classification

I have fitted a Gaussian mixture model to my data. This Gaussian mixture model is the combination of two Gaussian distributions. I call the first Gaussian distribution state 1 and the second Gaussian ...
0
votes
1answer
15 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
1
vote
1answer
111 views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
3
votes
0answers
21 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
1
vote
1answer
54 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
2
votes
1answer
110 views

Sampling from Gaussian mixture models, when are the sampled data independent?

Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions $p(x) = \sum\limits_{i = 1}^N w_i \mathcal{N}(x;\mu_i, \Sigma_i)$ where $w_i$ are the weights. Now I sample some points $\...
7
votes
2answers
648 views

Why use a Gaussian mixture model?

I am learning about Gaussian mixture models (GMM) but I am confused as to why anyone should ever use this algorithm. How is this algorithm better than other standard clustering algorithm such as $K$-...
0
votes
0answers
10 views

Imposing constraints on observation model in a HMM

I have $N$ observations ($x_1, x_2,.. ,x_N$) from a HMM with $K$ latent states. The M step for computing the observation model $\mu_k$ involves maximizing the expression: $$ L = \sum_{n=1}^{N}{ln \...
1
vote
1answer
14 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
0
votes
0answers
9 views

Can we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?

I learned from this answer that: A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A gaussian-mixture is the ...
0
votes
0answers
19 views

Redefining latent variables as observed data

This was just a thought that occurred to me, but technically, is it possible to redefine what I treat as latent variables and what I treat as data? For example, lets assume I have a set of latent ...
0
votes
0answers
23 views

Standard deviation in multimodal data

I have a dataset, 90% of observations are unimodal normal (with couple of outliers per feature), 10% are mixture of normals, components have the same standard deviations. Data contains outliers => ...
1
vote
1answer
27 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
0
votes
0answers
14 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
1
vote
0answers
34 views

Stopping criteria for gaussian mixture models

As I can read from the source code of scikit-learn, the stopping criteria for the iterative algorithm of Expectation Maximization (in my case applied to fitting Gaussian mixture models) is to put a ...
0
votes
0answers
24 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
2
votes
2answers
114 views

Compute quantile function from a mixture of Normal distribution

I have this mixture of normal distribution: $$X \sim \frac{1}{2}\mathcal{N}(\mu_{x_1}=10,\,\sigma_{x_1}^{2}=1)+\frac{1}{2}\mathcal{N}(\mu_{x_2}=13,\,\sigma_{x_2}^{2}=1)$$ How can i compute the ...
1
vote
1answer
36 views

Calculating the probability that an observation comes from either population A or B

If I have two normal distributions A (mean = 0, variance = 4) and B (mean = 0, variance = 16), how can I calculate the probability that an observation with magnitude 2 comes from A?
2
votes
1answer
35 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
1
vote
0answers
21 views

Tensorflow InvalidArgumentError: The determinant is not finite [closed]

I'm trying to fit a Mixture of Gaussians to a data set. First the data is clustered using K-Means Clustering. Each cluster is then fitted with a Gaussian.To avoid inversion of large covariance matrix, ...
2
votes
1answer
38 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
92 views

Gaussian Mixture: is this plot right?

I'm studying about Gaussian Mixtures and I decided to play around with it in Python, but I'm not entirely sure if I understand it fully. I generated some data, and then calculated the Gaussian ...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
0
votes
1answer
76 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
2
votes
0answers
29 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
0
votes
0answers
20 views

GMM model of the joint distribution from multivariate marginals

I have two multivariate Gaussian variables $X \sim \mathcal{N}(\boldsymbol {\mu}_X \in \mathcal R^d,\boldsymbol {\Sigma}_X \in \mathcal R^{d \times d})$ and $Y \sim \mathcal{N}(\boldsymbol {\mu}_Y \...
0
votes
1answer
88 views

Gaussian Mixture Model with k=n clusters

Suppose that we construct a Gaussian Mixture Model with FIXED COVARIANCE on $n$ points using $k=n$ clusters. Is it the case that the Maximum Likelihood parameters put each of the $n$ points in their ...
1
vote
1answer
84 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
2
votes
1answer
182 views

Scikit-Learn Gaussian Mixture: How can log-probabilities be positive? [closed]

I am fitting a Gaussian Mixture model: gm = GaussianMixture(n_components=K) gm.fit(features) When I do: ...
0
votes
1answer
65 views

Is there anything like “implied density” in this experiment?

A basket of balls is dropped into a maze. When a ball is dropped into the maze at the top it moves downward, pulled by gravity, through a series of nails. The ball then falls down to a new level where ...
1
vote
1answer
93 views

Reducibility between Gaussian Mixture Models and Gaussian Processes

I am studying gaussian processes and I have already discrete amount of knowledge in gaussian mixture models. I am here to undersrtand if with a gaussian process you can fit a gaussian mixture model. ...
1
vote
1answer
42 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
1
vote
1answer
68 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
0
votes
1answer
25 views

Concise way to visualize / compare many Gaussian mixtures

I have 5,000 samples drawn from each of approximately 50,000 distributions. I have good reason to expect most of them to be normally distributed, and I expect some of them to be multi-modal (mixture ...
0
votes
0answers
14 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
213 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
3
votes
1answer
88 views

How to cluster parts of broken line made of points?

I am studying clustering techniques and i am pretty new at this topic. Here is my problem: I created a 5 lines which are made of points. This lines are supposed to be continuous and they look like ...
3
votes
2answers
305 views

Gaussian mixture model - does an improper uniform prior give a proper posterior?

We draw $n$ i.i.d. points $x_1 , x_2 , ..., x_n$ from the following Gaussian mixture: $$p(x|\mu_1,\mu_2) = \frac{1}{2} \text{N} (x|\mu_1,1) + \frac{1}{2} \text{N} (x|\mu_2,1).$$ The prior is the ...
0
votes
1answer
65 views

Are Gaussian Mixture Models stochastic or deterministic?

Each time we generate a gmm model, we obtain slightly different clusters. Can we hence say gmm is stochastic? We obtain the same clusters if a random seed is set; does this mean given a random seed, ...
1
vote
0answers
12 views

Does mixture of sigmoids make sense given the theories about mixture of bernoullis?

Mixture of bernoullis is the combination of bernoulli distributions, which can be illustrated by the sampling process of K bags of D coins, here is a quick tutorial about it https://cedar.buffalo.edu/~...
1
vote
0answers
29 views

Determining number of components in mixtures of normal distributions with common mean

This is a pretty simple question, suppose we want to fit a mixture distribution of multivariate normals with common mean $$y_i \sim \sum_k \pi_k N(\mu, \Sigma_k)$$ What is the preferred approach for ...
1
vote
1answer
68 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
3
votes
0answers
39 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
0
votes
0answers
5 views

Guassian Mixture Classification: interpretating component x variable means matrix

The Guassian Mixture model output by mclust::Mclust() function has a $parameters$mean element which is a matrix with dimensions ...
1
vote
0answers
20 views

Is there any background for constraining covariances on fitting GMM?

On clustering data using GMM model, I often see the option to constrain covariances of each clustered GMM. For example, http://scikit-learn.org/0.16/auto_examples/mixture/plot_gmm_classifier.html ...
0
votes
0answers
59 views

Output Size of Mixture Density Networks

I am working on a neural network in which the final output layer will be a Mixture Density Network (MDN), but am confused about the shape of the values that final layer should return. In the paper in ...
2
votes
1answer
44 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
1
vote
2answers
601 views

Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?

I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like: Since this was ...