Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
1
vote
1answer
213 views

About variance mixture models and probability distributions

I was wondering if anyone knows a good resource to learn about variance mixture models ? My interest is in particular the normal variance mean mixture. I know what they mean with their definition of $...
1
vote
2answers
214 views

Whitening a mixture of Gaussians

Suppose I have a dataset that was sampled from a mixture of Gaussians: $$ X \sim \sum_i w_i \mathcal{N}(\mu_i, \Sigma_i)$$ Technically, I can center and then whiten $X$ so that it has zero mean and ...
1
vote
1answer
4k views

Plotting a gaussian mixture with pdf values >> 1 in MATLAB

From a given dataset X, I learn a 7-component gaussian mixture model using matlab's gmdistribution.fit: ...
1
vote
1answer
45 views

Can we use a mixture of normal distributions while optimising likelihood?

Let's assume that we generate some values by a mixture of two Gaussians. Now we want to find the parameters of the two Gaussians by likelihood maximisation. One good expect that the optimisation will ...
1
vote
1answer
659 views

Is it important to make a feature scaling before using Gaussian Mixture Model?

Is it important to make a feature scaling before using Gaussian Mixture Model? and why is it important while we are using probability in getting our clusters's parameters (mean and covariance matrix). ...
1
vote
2answers
165 views

Meaning of Gaussian mixture model parameters

I came across this question from a tutorial: Suppose we have observations $x_1$ , $x_2$ , $\ldots$, $x_n$ of a continuous r.v. $X$ known to be drawn from a “mixture” of $k$ Gaussian distributions. ...
1
vote
1answer
1k views

Assessing Gaussian mixture distribution by cross validation

I have a 10 dimensional random vector that I'm modelling with GMMs. I want to estimate the best number of mixtures ($K$) for my data via the following method: Divide the data to train (90%) and test (...
1
vote
2answers
813 views

Manually generate random sample in Gaussian mixture model

I want to generate (manually) a random sample in the Gaussian mixture model: $$f_{\theta}(x) = \sum_{k = 1}^{K}\pi_k f_{\mathcal N(\mu_k, \sigma^2_k)}(x)$$ Here is my work: ...
1
vote
1answer
944 views

How to know if my Gaussian mixture model has enough training data?

A somewhat soft question - I'm training a Gaussian mixture model (with the EM algorithm) on data of size $N$ ($N$ is typically between 4 and 64). How much samples do I need? obviously it depends on ...
1
vote
1answer
146 views

How to project data onto a model (specifically, GMM)?

I'm using data to train a Gaussian mixture model (GMM). I then take a sample and would like to see its projection on the GMM 'space'. I can think of an optimization problem such as this: consider $y$ ...
1
vote
1answer
420 views

Bayesian Gaussian Mixture model

I am trying to fit basic Gaussian mixture with a Bayesian model. My likelihood function is Gaussian, with std=1, and the only parameter is the mean, chosen from $\{0,1,\dots,14,15\}$ and my prior is ...
1
vote
1answer
557 views

Clustering data with Fourier series representation

We are analyzing temporal behavioral patterns across many users and we want to cluster users in order to understand "natural types of behavior". Our idea is to represent the data (672 bins for each ...
1
vote
1answer
991 views

How do I “split” Gaussian mixture components when training EM/GMM based classifier?

In order to improve performance of my Gaussian Mixture Model based classifier, I was recommended to start with a single multivariate Gaussian, estimate its parameters, and "split" it into two mixtures,...
1
vote
2answers
196 views

Mixture Model with dependant observations

I am trying to model a process in which each datapoint is generated sequentially, so the current observation depends on the last one. Some example data could look like, ...
1
vote
1answer
45 views

What is a mixing process?

What does this mean? Asset prices follow a mixture of normal distributions with a mixing process dependent on the unobservable information arrival process.
1
vote
1answer
717 views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
1
vote
1answer
17 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
1
vote
1answer
115 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
1
vote
1answer
97 views

Reducibility between Gaussian Mixture Models and Gaussian Processes

I am studying gaussian processes and I have already discrete amount of knowledge in gaussian mixture models. I am here to undersrtand if with a gaussian process you can fit a gaussian mixture model. ...
1
vote
1answer
102 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
1
vote
1answer
107 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
1
vote
1answer
92 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
1
vote
1answer
144 views

Gaussian mixture models with constrained mixing proportions

I am fitting a Gaussian mixture model to multivariate data and my application suggests constraining the mixing proportions to lie in a pre-determined sub-space. I am curious if such an approach has ...
1
vote
1answer
640 views

General conditional distributions for multivariate Gaussian mixtures

My question is similar to this one but considers a more general situation. Suppose that $ \vec{x} = (x_1, \dots, x_d) $ and let $$ p(\vec{x}) = \sum_{k=1}^{n} \pi_k \mathcal{N}(\vec{x} | \mu_k, \...
1
vote
1answer
219 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
1
vote
1answer
206 views

Conditional mean for mixture of multivariate normal distributions

If x = (x_1,x_2,...,x_n) is a vector whose components have a distribution that is a finite mixture of multivariate normals, is the expected value of x_1 still a linear function of the other components,...
1
vote
1answer
547 views

Estimating truncation point in Gaussian mixture

I have data modeled as a mixture of two Gaussian distributions. The data is "clipped" i.e., there is data only for values greater than a threshold $t$, even though it is feasible for data to exist in ...
1
vote
1answer
32 views

Mixture modelling of data with measurement uncertainty

I have a dataset that consists of a population radiometric ages (300>n>600). A dataset can have ages can range on the order of billions of years. Each age measurement has an associated uncertainty ...
1
vote
1answer
475 views

Why use EM algorithm instead of just plain old ML for mixture model?

Let's say I have some [multivariate] data and want to fit a GMM to it. So I have $P_x=\sum_{i=1}^{n}\alpha_i{N(x;\theta_i)}$, where $x$ is an observation from the data, $\theta_i$ is the mean and ...
1
vote
1answer
278 views

Gradient Ascent for Mixture of Gaussians

Beginner here, apologies if this is something very simple. I am trying to do a gradient ascent to estimate means for a Mixture of Gaussian model. I am using $(x-µ)/σ^3(2π)^{(1/2)} * e^{-((x-µ)/σ)^2)/2}...
1
vote
1answer
213 views

Generate data points for a Gaussian with drawing probability

I am trying to solve this question: Generate 500 data points drawn from each of 3 (three) Gaussians: $N_1(1, 0.1)$, $N_2(1.5., 0.1)$ and $N_3(2, 0.2)$ whose drawing probability on each iteration ...
1
vote
1answer
640 views

How can I find mean and covariance after EM iteration on GMM algorithmm?

I have a dataset divided in 2 class(lets call x1,x2) but I don't know their mean and covariance. For each class I looked their graph and made a guess about their sub-classes, then run an EM(...
1
vote
1answer
562 views

Understanding hidden markov model, and how it is applied in speech recognition

I have for some some time tried to understand how this hidden markov model (hmm) works, and have found a lot of tutorials/papers on it which make use of the same examples/principles of explaining the ...
1
vote
1answer
137 views

Learn parameters for truncated Gaussian

I would like to learn the parameters for a truncated gaussian like this one. I'm using this formula for the probability density $f(x | \mu, \sigma^2) = \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \...
1
vote
1answer
1k views

Derivation of M-step in EM algorithm for mixture of Gaussians

I am trying to derive the parameter estimation equations for the M-step of the expectation maximization (EM) algorithm for a mixture Gaussians when all Gaussians share the same covariance matrix $\...
1
vote
1answer
38 views

what is the difference in training and testing for Gaussian and Mixture of Gaussians

what is the difference in training and testing between the Gaussian and Mixture of Gaussians? Are they the same except one is unimodal and one is multimodal?
1
vote
1answer
439 views

Decomposition of multimodal distributions

I have decomposed a multimodal distribution into the constituent single distributions for for further analysis. I have spent some time researching various approaches and I have not found one that that ...
1
vote
1answer
196 views

QDA vs EM with Gaussian likelihoods

QDA (quadratic discriminant analysis) assumes that the K different classes are generated by K different multivariate Gaussians, each with potentially different mean vector and covariance matrix. If ...
1
vote
1answer
457 views

Understanding the E step of EM for GMM

I'm reading this chapter about EM (9.3.1) of the book "Pattern Recognition and Machine Learning". I understand the basic EM algorithm for GMM, but I'm having some problems understanding the ...
1
vote
1answer
716 views

Is the posterior distribution on means in a Bayesian Gaussian mixture model with symmetric priors Gaussian?

I am reading through a document on learning Gaussian mixture models in Infer.NET. They assume the data is generated from 2 Gaussians where the prior distribution on means is Gaussian and the prior ...
1
vote
1answer
99 views

Gaussian Mixture Model: bandwidth parameter versus variogram fitting?

I'm estimating a stationary, spatially random variable over a 2-dimensional domain. I have ground-truth measurements in several locations, over time. I need some way of spatially-interpolating ...
1
vote
1answer
65 views

Compression of gaussian mixtures

Is there a general way of fitting a gaussian mixture with $m$ components with a gaussian mixture with $n \ll m$ components, short of generating samples and redoing a full fit? I expect a lot of ...
1
vote
1answer
2k views

How does the number of components in a GMM relate to the information content?

Say you fit a Gaussian Mixture Model (GMM) to your data using a Bayesian technique, which should tell you the number of components needed to fit your data. Does this also give insight into the ...
1
vote
1answer
499 views

Is derivative of a Gaussian Signal also Gaussian? How to find variance of signal that is obtained from differentiation of a Gaussian signal?

Could someone please let me know or give appropriate references for the question I have posed above. My main interest lies in applying Kalman filter for state estimation. The noise on sensor ...
1
vote
2answers
293 views

Finding out if your data belongs to normal distribution

Is there a way to find out if your data belongs to one or more (mixture) normal distributions? I probably could calculate what is the standard deviation of my data, but I'm not sure what else to do ...
1
vote
1answer
771 views

GLM link function for bimodal probit fitting?

I am trying to model a set of data I have physical reason to believe can be represented by a bimodal normal cumulative distribution function (Technically it is a bimodal log-normal CDF, but I think I ...
1
vote
1answer
230 views

For inference of Dirichlet Process Mixture, why the expected value $\int h(x)f(x)$ is desired?

Why the expected value $\int h(x)f(x)$ is desired for inference in Dirichlet Process Mixture? What is the intuition for MCMC in Dirichlet Process Mixture? $f(x)$ is the probability density function, ...
1
vote
1answer
410 views

Relation between Gaussian mixture models and maximum likelihood?

I need some help understanding the relation between the maximum likelihood and Gaussian mixture models. I have seen that there is a relationship between the expectation maximization algorithms and ...
1
vote
1answer
764 views

Bayes' theorem in 1-d EM algorithm

I'm watching a video on the EM algorithm, When we use Bayes' Theorem to calculate $b_i$, how do I find $P(b)$ and $P(a)$ initially? It says we can estimate the priors $P(b)$ and $P(a)$ but that's ...
1
vote
1answer
303 views

Efficience of Expectation-Maximization algorithm in function of learning dataset size

I have datasets of increasing sizes identically distributed. I have tried to fit a gaussian mixture to these datasets using Expectation-Maximization algorithm. To check the quality of this fit, I ...