Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

3
votes
0answers
813 views

Fit Gaussian Mixture model directly to the mixture density

The core of the question is: Can I estimate the parameters of a gaussian mixture model (with EM or Dirichlet Process) from a mixture density directly, that is, without using data drawn from such ...
3
votes
0answers
452 views

How to calculate BIC for multidimensional problem

Sorry for this question, but I am really not sure how to calculate BIC for my situation. My models are mixtures of normals with different number of components. Variances are equal for all components ...
3
votes
0answers
363 views

Diffusion coefficient from double-normal probability density function

The spread of individuals of species is often described by so-called dispersal kernels. The main parameter of spread is then often the variance defined as the average squared movement distance of a ...
3
votes
0answers
223 views

Why isn't a gaussian mixture prone to overfitting?

Consider a Gaussian mixture of 2 components and a dataset of size $N$. The EM algorithm use the data to estimate: the model parameters: the means $\mu_1, \mu_2$ (say the covariances matrices are ...
3
votes
2answers
2k views

Maximum likelihood estimation for Gaussian mixture

When doing maximum likelihood (ML) inference on a Gaussian mixture model (GMM), Bishop notes in PRML that if there is more than one mixture component in the GMM and the mean of one Gaussian collapses ...
3
votes
0answers
428 views

A better bayesian way of modelling autoregressive mixtures

I have a JAGS hierarchical model which includes a temporal sub-model for the primary vote share between four party groups (LNP, Labor, Green, and Other). For each day in the temporal model, the vote ...
2
votes
2answers
260 views

AICc is picking overly complex models - something stricter?

I'd like to know if there are stricter alternatives to automated model selection than AICc / AIC / BIC. We have approximately ten thousand curves, and for each we'd like to find the most parsimonious ...
2
votes
1answer
2k views

Convergence of EM for Mixture of Gaussians

Is the Mixture of Gaussians model (an example of latent class analysis) gauranteed to converge on a viable solution even on Unimodal data using the Expectation Maximization algorithm to estimate the ...
2
votes
1answer
156 views

Sampling from Gaussian mixture models, when are the sampled data independent?

Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions $p(x) = \sum\limits_{i = 1}^N w_i \mathcal{N}(x;\mu_i, \Sigma_i)$ where $w_i$ are the weights. Now I sample some points $\...
2
votes
1answer
120 views

Gaussian Mixture: is this plot right?

I'm studying about Gaussian Mixtures and I decided to play around with it in Python, but I'm not entirely sure if I understand it fully. I generated some data, and then calculated the Gaussian ...
2
votes
1answer
742 views

A Gaussian Mixture Model Is a Universal Approximator of Densities

When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following: A Gaussian mixture model is a universal approximator of densities, in the ...
2
votes
1answer
260 views

Scikit-Learn Gaussian Mixture: How can log-probabilities be positive? [closed]

I am fitting a Gaussian Mixture model: gm = GaussianMixture(n_components=K) gm.fit(features) When I do: ...
2
votes
1answer
239 views

Doesn't the non-Gaussian source assumption of ICA render it practically useless?

Gaussian distributions appear everywhere in nature, indeed this was largely the justification for most classical methods' reliance on assumption of normality. ICA assumes non-Gaussian sources, indeed ...
2
votes
2answers
806 views

Which metric is used in the EM algorithm for GMM training ?

My question concerns the expectation-maximisation algorithm used to estimate the hyper-parameters of a Gaussian mixture model in z multivariate setup. I understand that the EM algorithm uses the ...
2
votes
1answer
831 views

When would I use EM instead of k-means?

When would I want to assign cluster probabilities to patterns instead of hard assignments to clusters? Can someone elaborate?
2
votes
1answer
268 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
2
votes
3answers
1k views

What's the difference between Multivariate Gaussian and Mixture of Gaussians?

What's the difference between Multivariate Gaussian and Mixture of Gaussians? If I have a Multivariate Gaussian and making all the data into ONE vector, is that a Mixture of Gaussians in 1 dimension?...
2
votes
2answers
394 views

Why are hidden Markov models considered 'mixture models'?

I'm a bit confused by the conception of "mixture model" I'm studying hidden Markov model, which is frequently referred to as a "mixture model". But I don't know what the term "mixture" implies. ...
2
votes
2answers
2k views

What is the best function to fit onto a “flat top gaussian”?

I am studying some signal and I am trying to make an automated algorithm that can extract the parameters for my signal. First let me describe a bit, I have a light emiting object crossing a window of ...
2
votes
1answer
2k views

Gaussian Process Regression With Multiple Inputs?

Is it possible to use a Gaussian Process to relate multiple independent input variables (X1, X2, X3) to an output variable (Y)? More specifically, I would like to produce a regression graph like the ...
2
votes
2answers
188 views

Compute quantile function from a mixture of Normal distribution

I have this mixture of normal distribution: $$X \sim \frac{1}{2}\mathcal{N}(\mu_{x_1}=10,\,\sigma_{x_1}^{2}=1)+\frac{1}{2}\mathcal{N}(\mu_{x_2}=13,\,\sigma_{x_2}^{2}=1)$$ How can i compute the ...
2
votes
1answer
2k views

Sampling from a multivariate Gaussian mixture model [duplicate]

How can I generate multi-dimensional data from a (estimated) Gaussian mixture pdf? In general, what would be ways to generate multi-dimensional data from a pdf? I read rejection sampling can be used, ...
2
votes
1answer
2k views

Differences linear discriminant analysis and Gaussian mixture model

I know that there are topics about this question but in my view, the answers are not clear enough. I don't understand the main difference between Linear Discriminant Analysis (LDA) and Gaussian ...
2
votes
1answer
160 views

Gaussian mixture distribution confusion

Have some confusion on Gaussian mixture distribution model (reference is from the book of Pattern Recognition and Machine Learning). My confusion is how below formula works? $\sum_kz_k=1$ My thought ...
2
votes
2answers
177 views

Applying Akaike Information Criterion on collection of Gaussian fits

I am trying to apply Akaike Information Criterion on a collection of Gaussian mixtures fitted on some data points. My question is, can I use AIC even if the number of components of Gaussian mixtures ...
2
votes
1answer
956 views

Time varying Gaussian distribution

Does there exist time varying Gaussian model? To be specific, a 2D Gaussian model whose mean and covariance matrix is varying with another parameter called time. The $\mu$(mean) and $\sigma$(...
2
votes
1answer
462 views

Convert normal pdf into lognormal pdf

I have fitted a mixture of 3 normal distributions to my log-transformed data Y, as the package I'm using cannot fit a mixture of lognormal distributions. My questions is: how I can convert pdf from 3 ...
2
votes
1answer
326 views

Is there other types of mixture distribution besides the normal mixture

There are quite a lot of study on the normal mixture distributions, say, $X=Y*Z$,where $Z$ is a normal r.v. and Y is a r.v. follows other distributions and $Y$ and $Z$ are independent. Some well-known ...
2
votes
1answer
229 views

Best approach to classifying 3-point trajectories?

I have a sample of about 300 subjects who have been measured at 3 different times (morning, afternoon, evening). The variable of interest can be assumed to be approximately normal. It appears that ...
2
votes
2answers
2k views

Algorithms for 1D Gaussian mixture with equal variance and noise cluster

I would like to fit a Gaussian mixture model to some data. The data is 1D and I want to constrain all the Gaussians to have equal variance. I would also like to have a uniform background noise cluster ...
2
votes
1answer
908 views

comparing gaussian mixture models with and without regularization

I am using Gaussian mixture models for clustering a bunch of data sets. On some data sets a regularization value is often used/suggested (see the MATLAB example for Fisher's Iris data which sets ...
2
votes
1answer
282 views

Need intuition - how do they simplify the Q function for gaussian mixture EM?

Background - I'm trying to follow section 7.2.4 in this EM tutorial. Basically the setup is I have a vector with 10 points $x$, and each of them can be assigned ($y$) to either Gaussian 1 or to ...
2
votes
1answer
553 views

Defining overlapping periods

I have a dataset containing the abundance of migrating bird species. In the figure below you can see that there are two "bell" shapes that are overlapping somewhere around September. One of the bell ...
2
votes
1answer
188 views

GMM weighting change during marginalization

Suppose one has a multivariate Gaussian Mixture Model: $$ \text{pdf}(\vec{x}) = \sum_{i=1}^N w_i \mathcal{N}(\mu^{(i)}, \Sigma^{(i)}) $$ Suppose $\vec{x} = \{\vec{a},\vec{b}\}$ and we marginalize ...
2
votes
2answers
179 views

Gaussian Mixture, sampling and interpretation

I came across the following which I have troubles understanding and would appreciate your help. Suppose we want to fit a Gaussian mixture model (GMM) to a $t$-dstribution with some degrees of ...
2
votes
1answer
3k views

Difference between multivariate Gaussian distribution and multivariate Gaussian mixture model

I would like to know the difference between a multivariate Gaussian distribution and multivariate Gaussian mixture model. Can someone provide an intuitive and/or detailed explanation? Thanks.
2
votes
1answer
161 views

Gaussian Mixture and K-Means ?! a big challenge?

This is taken from Tom. Mitche Material as Old-Exam. I think the (2) is true and not (3). Who can verify me?
2
votes
2answers
87 views

Re-estimating a probability distribution with additional priors

I have a 3D dataset with at least millions of data points (scatter events from atoms, approximately Gaussian). I am modeling this data with a Gaussian Mixture Model. The usual approach would be to ...
2
votes
1answer
2k views

matlab gmdistribution.fit 'Regularize' - what regularization method?

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
2
votes
1answer
87 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
2
votes
1answer
39 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
2
votes
1answer
45 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
48 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
2
votes
1answer
113 views

Correlated random variables from mixture distributions

Let I have three random variables whose density is a mixture of two Normals with these parameters: $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$ $\...
2
votes
1answer
87 views

Introduction to Gaussian mixture models

First of all, I am sorry if this question is not acceptable by some of the readers. However, I really read many, many sources about Gaussian mixture models, but all what I found was a short tutorial ...
2
votes
1answer
332 views

Gaussian Mixture Model

with the following code I fit a Gaussian Mixture Model to arbitrarily created data. The code is working. The only thing I encounter is that during the calculation of the multivariate_normal I ...
2
votes
2answers
206 views

This is Gaussian mixture model?

Here is a problem that I am looking at. Is this model really commonly known as a Gaussian mixture model (the one often appears as an illustration of EM algorithm)? I am confused because Gaussian ...
2
votes
1answer
443 views

Gaussian Mixture Model - marginal likelihood

I am studying gaussian mixture models. The first step defines the following equation. They then proceed to marginalize $z_n$ out My question is, how did they arrive at that equation ? Where did the ...
2
votes
1answer
316 views

Use gradient methods for maximum likelihood estimation of Gaussian mixture

I have some questions concerning estimating maximum likelihood of Gaussian mixture model. As I have read around some material, they usually use EM algorithm for maximizing the complete likelihood ...
2
votes
1answer
915 views

How to deal with categorical feature in a Gaussian Mixture model clustering model

I am performing clustering by Gaussian Mixture model using EM algorithm in R. U use the mclust package. My data (205 observations and 25 variables) has both ...