Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
2
votes
1answer
292 views

Best approach to classifying 3-point trajectories?

I have a sample of about 300 subjects who have been measured at 3 different times (morning, afternoon, evening). The variable of interest can be assumed to be approximately normal. It appears that ...
2
votes
2answers
245 views

Whitening a mixture of Gaussians

Suppose I have a dataset that was sampled from a mixture of Gaussians: $$ X \sim \sum_i w_i \mathcal{N}(\mu_i, \Sigma_i)$$ Technically, I can center and then whiten $X$ so that it has zero mean and ...
2
votes
2answers
2k views

Algorithms for 1D Gaussian mixture with equal variance and noise cluster

I would like to fit a Gaussian mixture model to some data. The data is 1D and I want to constrain all the Gaussians to have equal variance. I would also like to have a uniform background noise cluster ...
2
votes
2answers
55 views

EM-algorithm for two clusters (when one of the distributions is uniform)

I am having a hard time with the EM-algorithm. Here's the problem that I am trying to solve. Dealing with noisy annotations is a common problem in computer vision, especially when using ...
2
votes
1answer
183 views

Distribution of Distributions | percent point function

I have the following normal distribution (Primary Distribution): Mean = 7 Sigma = 0.5 and a list of other normal distributions (Secondary Distributions): (python list of tupels, each tupel ...
2
votes
2answers
61 views

Can Gaussian Mixture Model Clustering tell me something about the distribution of my data?

I have 10,000 vectors originating from 5 separate classes (2,000 each). I use Gaussian Mixture Model clustering (in Python) to cluster the 10,000 vectors, telling the algorithm to cluster the data ...
2
votes
1answer
1k views

comparing gaussian mixture models with and without regularization

I am using Gaussian mixture models for clustering a bunch of data sets. On some data sets a regularization value is often used/suggested (see the MATLAB example for Fisher's Iris data which sets ...
2
votes
1answer
2k views

Assessing Gaussian mixture distribution by cross validation

I have a 10 dimensional random vector that I'm modelling with GMMs. I want to estimate the best number of mixtures ($K$) for my data via the following method: Divide the data to train (90%) and test (...
2
votes
1answer
344 views

Need intuition - how do they simplify the Q function for gaussian mixture EM?

Background - I'm trying to follow section 7.2.4 in this EM tutorial. Basically the setup is I have a vector with 10 points $x$, and each of them can be assigned ($y$) to either Gaussian 1 or to ...
2
votes
1answer
645 views

Defining overlapping periods

I have a dataset containing the abundance of migrating bird species. In the figure below you can see that there are two "bell" shapes that are overlapping somewhere around September. One of the bell ...
2
votes
1answer
1k views

How to know if my Gaussian mixture model has enough training data?

A somewhat soft question - I'm training a Gaussian mixture model (with the EM algorithm) on data of size $N$ ($N$ is typically between 4 and 64). How much samples do I need? obviously it depends on ...
2
votes
1answer
225 views

GMM weighting change during marginalization

Suppose one has a multivariate Gaussian Mixture Model: $$ \text{pdf}(\vec{x}) = \sum_{i=1}^N w_i \mathcal{N}(\mu^{(i)}, \Sigma^{(i)}) $$ Suppose $\vec{x} = \{\vec{a},\vec{b}\}$ and we marginalize ...
2
votes
2answers
273 views

Gaussian Mixture, sampling and interpretation

I came across the following which I have troubles understanding and would appreciate your help. Suppose we want to fit a Gaussian mixture model (GMM) to a $t$-dstribution with some degrees of ...
2
votes
1answer
467 views

Bayesian Gaussian Mixture model

I am trying to fit basic Gaussian mixture with a Bayesian model. My likelihood function is Gaussian, with std=1, and the only parameter is the mean, chosen from $\{0,1,\dots,14,15\}$ and my prior is ...
2
votes
1answer
828 views

Clustering data with Fourier series representation

We are analyzing temporal behavioral patterns across many users and we want to cluster users in order to understand "natural types of behavior". Our idea is to represent the data (672 bins for each ...
2
votes
1answer
164 views

Gaussian Mixture and K-Means ?! a big challenge?

This is taken from Tom. Mitche Material as Old-Exam. I think the (2) is true and not (3). Who can verify me?
2
votes
2answers
91 views

Re-estimating a probability distribution with additional priors

I have a 3D dataset with at least millions of data points (scatter events from atoms, approximately Gaussian). I am modeling this data with a Gaussian Mixture Model. The usual approach would be to ...
2
votes
2answers
4k views

Gaussian Naive Bayes really equivalent to GMM with diagonal covariance matrices?

Murphy writes that a multivariate Gaussian used in a generative classifier ("Gaussian discriminant analysis"), i.e., $p(\mathbf x|y=c,\mathbf\theta) = \mathcal{N}(\mathbf x|\mathbf y_c,\mathbf \...
2
votes
1answer
2k views

matlab gmdistribution.fit 'Regularize' - what regularization method?

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
2
votes
1answer
109 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
2
votes
1answer
50 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
2
votes
1answer
30 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
2
votes
1answer
44 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
2
votes
1answer
130 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
68 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
2
votes
1answer
165 views

Introduction to Gaussian mixture models

First of all, I am sorry if this question is not acceptable by some of the readers. However, I really read many, many sources about Gaussian mixture models, but all what I found was a short tutorial ...
2
votes
1answer
643 views

Gaussian Mixture Model

with the following code I fit a Gaussian Mixture Model to arbitrarily created data. The code is working. The only thing I encounter is that during the calculation of the multivariate_normal I ...
2
votes
2answers
251 views

This is Gaussian mixture model?

Here is a problem that I am looking at. Is this model really commonly known as a Gaussian mixture model (the one often appears as an illustration of EM algorithm)? I am confused because Gaussian ...
2
votes
1answer
680 views

Gaussian Mixture Model - marginal likelihood

I am studying gaussian mixture models. The first step defines the following equation. They then proceed to marginalize $z_n$ out My question is, how did they arrive at that equation ? Where did the ...
2
votes
1answer
760 views

Use gradient methods for maximum likelihood estimation of Gaussian mixture

I have some questions concerning estimating maximum likelihood of Gaussian mixture model. As I have read around some material, they usually use EM algorithm for maximizing the complete likelihood ...
2
votes
1answer
195 views

Useful separation value in a mixture distribution

Assume we have a distribution that is the mixture of two normal distributions. The pdf of the overall distributions and their single parts may look like the following. In black, the combined ...
2
votes
1answer
522 views

Mixture Density Network: What is C?

I'm currently trying to implement a Mixture Density Network (MDN) based off of the original paper here. Most of the equations seem pretty straight forward but on page 6 (7 of the PDF) equation 23 has ...
2
votes
2answers
661 views

The pointwise product of densities of a Gaussian mixuture and a Gaussian

Let's say that I have a mixture of Gaussians representing a likelihood: $$ p(\mathbf{x}) = \sum_{i=1}^K\phi_i \mathcal{N}(\boldsymbol{\mu_i,\Sigma_i}) $$ What is the posterior distribution given a ...
2
votes
1answer
464 views

One-class Classification of multidimensional vectors

I have a m x k User-feature matrix (m >> k) obtained by factorizing an original User-websites matrix (m x n) that has #page views as entries. Additionally, there are users (say r) who have been ...
2
votes
2answers
362 views

How to accomplish unsupervised separation of subpopulations?

I have a dataset drawn from a social network that looks Bimodal on logarithmic scales for all attributes (I'll demonstrate only one here): I know the variable that would give me a clean separation ...
2
votes
1answer
3k views

Number of free parameters in Gaussian mixture models

When comparing GMM models with different number of components (i.e number of Gaussians) one penalizes the likelihood for the total number of free parameters in the mixture model. If the data is in $D$ ...
2
votes
1answer
2k views

Implementing Gaussian mixture model for a HMM library

I'm working on an alignment algorithm using LAMP HMM library. This library supports Gaussian probability distribution but it does not seem to support Gaussian Mixture Model. What I want is, to input ...
2
votes
1answer
759 views

GMM with Bayes decision model

Given two classes of training data (A and B), I want to fit each class' distribution using a GMM with k components, and then use Bayes Decision Model for the classification. The first step was to ...
2
votes
0answers
49 views

testing whether data comes from a bi-modal distribution (python) [duplicate]

I have a variable which seems to be a mix of two Gaussian distributions (it is bi-modal with each mode looking normally distributed). I would like to identify anomalous samples. So my idea is to ...
2
votes
0answers
73 views

Can someone verify if the following Bayesian Information Criterion (BIC) model selection algorithm is correct for Gaussian mixture models?

I am trying to find an automated way of picking the number of clusters $K \in \mathbb{N}$ for unsupervised learning scenarios, specifically for GMM. I was suggested to use something called the "...
2
votes
0answers
77 views

Graphical model of the Gaussian mixture: where is n?

TL;DR: Where are the occupation numbers in the Graphical model of the GMM? I am implementing a Finite (to be adapted to infinite later) Gaussian Mixture Model. I am using the Gibbs sampler-ready ...
2
votes
0answers
125 views

Efficient sampling from a multivariate Gaussian Mixture distribution for a given CDF level

I have a multivariate Gaussian Mixture (GM) distribution. I am wondering if there is any more efficient way of drawing samples (i.e., identify the iso-surface) from a multivariate Gaussian Mixture ...
2
votes
1answer
486 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
2
votes
0answers
46 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
2
votes
0answers
101 views

Statistical test for comparing two-gaussian mixture

I have a distribution of shape sizes under two different (biological) conditions. From prior knowledge, I do expect there to be two populations. I fit each condition to a two-Gaussian mixture model. ...
2
votes
0answers
27 views

In a normal mixture model, what general happens if the number of groups to be summed far exceeds the dimensions of each normal?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_kI_d) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean vector $\mu$ and d-...
2
votes
0answers
37 views

Multimodality of mixtures of more than two Normal distributions

Let $$\phi(x;\mu,\sigma) = \frac{1}{\sigma \sqrt{2\pi}} \exp \left(- \frac{(x-\mu)^2}{2\sigma^2}\right)$$ denote the Gaussian density function ($\sigma > 0$). Let $$f(x) = \sum_{i=1}^N p_i \...
2
votes
3answers
731 views

GMM EM algorithm complexity per iteration

I was fitting GMM clusters with diagonal covariance on my data using EM with $n$ (=5e6) points, each having $m$ (=160) ...
2
votes
0answers
81 views

Gaussian Mixture Division

In the study of probabilistic graphical models (PGMs), the loopy belief update propagation (LBUP) message passing algorithm requires the division of unnormalised probability distributions. If the ...
2
votes
0answers
68 views

Distribution of the initializing set at K-means++

There is a well-known modification of the initializing step of K-means, named K-means++. It chooses cluster centers with probability proportional to its squared distance from the point's closest ...

1 2 3
4
5
11