Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

1
vote
2answers
155 views

Meaning of Gaussian mixture model parameters

I came across this question from a tutorial: Suppose we have observations $x_1$ , $x_2$ , $\ldots$, $x_n$ of a continuous r.v. $X$ known to be drawn from a “mixture” of $k$ Gaussian distributions. ...
1
vote
1answer
1k views

Assessing Gaussian mixture distribution by cross validation

I have a 10 dimensional random vector that I'm modelling with GMMs. I want to estimate the best number of mixtures ($K$) for my data via the following method: Divide the data to train (90%) and test (...
1
vote
2answers
749 views

Manually generate random sample in Gaussian mixture model

I want to generate (manually) a random sample in the Gaussian mixture model: $$f_{\theta}(x) = \sum_{k = 1}^{K}\pi_k f_{\mathcal N(\mu_k, \sigma^2_k)}(x)$$ Here is my work: ...
1
vote
1answer
895 views

How to know if my Gaussian mixture model has enough training data?

A somewhat soft question - I'm training a Gaussian mixture model (with the EM algorithm) on data of size $N$ ($N$ is typically between 4 and 64). How much samples do I need? obviously it depends on ...
1
vote
1answer
136 views

How to project data onto a model (specifically, GMM)?

I'm using data to train a Gaussian mixture model (GMM). I then take a sample and would like to see its projection on the GMM 'space'. I can think of an optimization problem such as this: consider $y$ ...
1
vote
1answer
414 views

Bayesian Gaussian Mixture model

I am trying to fit basic Gaussian mixture with a Bayesian model. My likelihood function is Gaussian, with std=1, and the only parameter is the mean, chosen from $\{0,1,\dots,14,15\}$ and my prior is ...
1
vote
1answer
508 views

Clustering data with Fourier series representation

We are analyzing temporal behavioral patterns across many users and we want to cluster users in order to understand "natural types of behavior". Our idea is to represent the data (672 bins for each ...
1
vote
1answer
966 views

How do I “split” Gaussian mixture components when training EM/GMM based classifier?

In order to improve performance of my Gaussian Mixture Model based classifier, I was recommended to start with a single multivariate Gaussian, estimate its parameters, and "split" it into two mixtures,...
1
vote
2answers
188 views

Mixture Model with dependant observations

I am trying to model a process in which each datapoint is generated sequentially, so the current observation depends on the last one. Some example data could look like, ...
1
vote
1answer
45 views

What is a mixing process?

What does this mean? Asset prices follow a mixture of normal distributions with a mixing process dependent on the unobservable information arrival process.
1
vote
2answers
211 views

Whitening a mixture of Gaussians

Suppose I have a dataset that was sampled from a mixture of Gaussians: $$ X \sim \sum_i w_i \mathcal{N}(\mu_i, \Sigma_i)$$ Technically, I can center and then whiten $X$ so that it has zero mean and ...
1
vote
1answer
388 views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
1
vote
1answer
15 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
1
vote
1answer
27 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
1
vote
1answer
104 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
1
vote
1answer
94 views

Reducibility between Gaussian Mixture Models and Gaussian Processes

I am studying gaussian processes and I have already discrete amount of knowledge in gaussian mixture models. I am here to undersrtand if with a gaussian process you can fit a gaussian mixture model. ...
1
vote
1answer
96 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
1
vote
1answer
87 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
1
vote
1answer
137 views

Gaussian mixture models with constrained mixing proportions

I am fitting a Gaussian mixture model to multivariate data and my application suggests constraining the mixing proportions to lie in a pre-determined sub-space. I am curious if such an approach has ...
1
vote
1answer
560 views

General conditional distributions for multivariate Gaussian mixtures

My question is similar to this one but considers a more general situation. Suppose that $ \vec{x} = (x_1, \dots, x_d) $ and let $$ p(\vec{x}) = \sum_{k=1}^{n} \pi_k \mathcal{N}(\vec{x} | \mu_k, \...
1
vote
1answer
203 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
1
vote
1answer
192 views

Conditional mean for mixture of multivariate normal distributions

If x = (x_1,x_2,...,x_n) is a vector whose components have a distribution that is a finite mixture of multivariate normals, is the expected value of x_1 still a linear function of the other components,...
1
vote
1answer
537 views

Estimating truncation point in Gaussian mixture

I have data modeled as a mixture of two Gaussian distributions. The data is "clipped" i.e., there is data only for values greater than a threshold $t$, even though it is feasible for data to exist in ...
1
vote
1answer
32 views

Mixture modelling of data with measurement uncertainty

I have a dataset that consists of a population radiometric ages (300>n>600). A dataset can have ages can range on the order of billions of years. Each age measurement has an associated uncertainty ...
1
vote
1answer
468 views

Why use EM algorithm instead of just plain old ML for mixture model?

Let's say I have some [multivariate] data and want to fit a GMM to it. So I have $P_x=\sum_{i=1}^{n}\alpha_i{N(x;\theta_i)}$, where $x$ is an observation from the data, $\theta_i$ is the mean and ...
1
vote
1answer
251 views

Gradient Ascent for Mixture of Gaussians

Beginner here, apologies if this is something very simple. I am trying to do a gradient ascent to estimate means for a Mixture of Gaussian model. I am using $(x-µ)/σ^3(2π)^{(1/2)} * e^{-((x-µ)/σ)^2)/2}...
1
vote
1answer
211 views

Generate data points for a Gaussian with drawing probability

I am trying to solve this question: Generate 500 data points drawn from each of 3 (three) Gaussians: $N_1(1, 0.1)$, $N_2(1.5., 0.1)$ and $N_3(2, 0.2)$ whose drawing probability on each iteration ...
1
vote
1answer
593 views

How can I find mean and covariance after EM iteration on GMM algorithmm?

I have a dataset divided in 2 class(lets call x1,x2) but I don't know their mean and covariance. For each class I looked their graph and made a guess about their sub-classes, then run an EM(...
1
vote
1answer
524 views

Understanding hidden markov model, and how it is applied in speech recognition

I have for some some time tried to understand how this hidden markov model (hmm) works, and have found a lot of tutorials/papers on it which make use of the same examples/principles of explaining the ...
1
vote
1answer
122 views

Learn parameters for truncated Gaussian

I would like to learn the parameters for a truncated gaussian like this one. I'm using this formula for the probability density $f(x | \mu, \sigma^2) = \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \...
1
vote
1answer
1k views

Derivation of M-step in EM algorithm for mixture of Gaussians

I am trying to derive the parameter estimation equations for the M-step of the expectation maximization (EM) algorithm for a mixture Gaussians when all Gaussians share the same covariance matrix $\...
1
vote
1answer
38 views

what is the difference in training and testing for Gaussian and Mixture of Gaussians

what is the difference in training and testing between the Gaussian and Mixture of Gaussians? Are they the same except one is unimodal and one is multimodal?
1
vote
1answer
412 views

Decomposition of multimodal distributions

I have decomposed a multimodal distribution into the constituent single distributions for for further analysis. I have spent some time researching various approaches and I have not found one that that ...
1
vote
1answer
190 views

QDA vs EM with Gaussian likelihoods

QDA (quadratic discriminant analysis) assumes that the K different classes are generated by K different multivariate Gaussians, each with potentially different mean vector and covariance matrix. If ...
1
vote
1answer
436 views

Understanding the E step of EM for GMM

I'm reading this chapter about EM (9.3.1) of the book "Pattern Recognition and Machine Learning". I understand the basic EM algorithm for GMM, but I'm having some problems understanding the ...
1
vote
1answer
703 views

Is the posterior distribution on means in a Bayesian Gaussian mixture model with symmetric priors Gaussian?

I am reading through a document on learning Gaussian mixture models in Infer.NET. They assume the data is generated from 2 Gaussians where the prior distribution on means is Gaussian and the prior ...
1
vote
1answer
94 views

Gaussian Mixture Model: bandwidth parameter versus variogram fitting?

I'm estimating a stationary, spatially random variable over a 2-dimensional domain. I have ground-truth measurements in several locations, over time. I need some way of spatially-interpolating ...
1
vote
1answer
60 views

Compression of gaussian mixtures

Is there a general way of fitting a gaussian mixture with $m$ components with a gaussian mixture with $n \ll m$ components, short of generating samples and redoing a full fit? I expect a lot of ...
1
vote
1answer
2k views

How does the number of components in a GMM relate to the information content?

Say you fit a Gaussian Mixture Model (GMM) to your data using a Bayesian technique, which should tell you the number of components needed to fit your data. Does this also give insight into the ...
1
vote
1answer
491 views

Is derivative of a Gaussian Signal also Gaussian? How to find variance of signal that is obtained from differentiation of a Gaussian signal?

Could someone please let me know or give appropriate references for the question I have posed above. My main interest lies in applying Kalman filter for state estimation. The noise on sensor ...
1
vote
2answers
285 views

Finding out if your data belongs to normal distribution

Is there a way to find out if your data belongs to one or more (mixture) normal distributions? I probably could calculate what is the standard deviation of my data, but I'm not sure what else to do ...
1
vote
1answer
755 views

GLM link function for bimodal probit fitting?

I am trying to model a set of data I have physical reason to believe can be represented by a bimodal normal cumulative distribution function (Technically it is a bimodal log-normal CDF, but I think I ...
1
vote
1answer
228 views

For inference of Dirichlet Process Mixture, why the expected value $\int h(x)f(x)$ is desired?

Why the expected value $\int h(x)f(x)$ is desired for inference in Dirichlet Process Mixture? What is the intuition for MCMC in Dirichlet Process Mixture? $f(x)$ is the probability density function, ...
1
vote
1answer
407 views

Relation between Gaussian mixture models and maximum likelihood?

I need some help understanding the relation between the maximum likelihood and Gaussian mixture models. I have seen that there is a relationship between the expectation maximization algorithms and ...
1
vote
1answer
755 views

Bayes' theorem in 1-d EM algorithm

I'm watching a video on the EM algorithm, When we use Bayes' Theorem to calculate $b_i$, how do I find $P(b)$ and $P(a)$ initially? It says we can estimate the priors $P(b)$ and $P(a)$ but that's ...
1
vote
1answer
299 views

Efficience of Expectation-Maximization algorithm in function of learning dataset size

I have datasets of increasing sizes identically distributed. I have tried to fit a gaussian mixture to these datasets using Expectation-Maximization algorithm. To check the quality of this fit, I ...
1
vote
1answer
30 views

how can I concatenate these mixtures

I have a matrix. for each row of this matrix I made a Gaussian mixture, how can I concatenate these mixtures.
1
vote
2answers
421 views

Maximum likelihood estimation get biased by fat-tail samples identifying mixture model of 2 normal distributions

I'm building a mixture model consists 2 normal distribution: \begin{equation} X_1 \sim \mathcal{N}(\mu_1,\sigma_{1}^{2}), \quad X_2 \sim \mathcal{N}(\mu_2,\sigma_{2}^{2})\end{equation} while $$pdf(...
1
vote
1answer
510 views

EM algorithm for Gaussian mix

Can anyone help me with the R code to implement EM algorithm. I got different value if I chose different starting value; clearly this is not good. And the value of $\mu$, $\sigma$ goes to NA after ...
1
vote
1answer
1k views

When is Gaussian mixture model more effective than Single Gaussian?

From my experiment, given training samples of around 600 each having vector length of 30, the single gaussian component performed much better unexpectedly. Having said that, I want to know how or when ...