Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

163 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
2answers
807 views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
4
votes
0answers
32 views

Implementation of EM algorithm confusion

Here EM algorithm manually implemented, there's a question of the implementation in R of the EM algorithm for 2 mixed gaussians. The answer has a supposedly correct implementation. However, don't the ...
4
votes
0answers
1k views

Finite mixture models with bounded data

I am trying to fit a finite mixture model to a dependent variable which is bounded (practically) between -0.594 and 1 (theoretically, the latent variable is bounded between -Inf - 1). The data are ...
4
votes
0answers
3k views

Ftting a mixture of two Gaussians

I want to fit a mixture of two gaussian densities to my financial data. The data can be found here: http://uploadeasy.net/upload/2a7mw.rar the variable is called dat. The probability density of a ...
3
votes
0answers
36 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
3
votes
0answers
50 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
3
votes
0answers
33 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
3
votes
0answers
696 views

Correct number of components in GMM according to BIC and AIC plots

I have applied GMM(Gaussian Mixture Model) to my data set and I have plotted the resulting BIC(Bayesian Information Criterion) and AIC(Akaike Information Criterion) for different number of components. ...
3
votes
2answers
227 views

Universal Approximation Capabilities of Mixture Models

I am looking for two reference incl. proofs showing 1) that a discrete Mixture of Gaussians can asymptotically approximate any (well behaved) continuous density 2) that a discrete Mixture of ...
3
votes
0answers
203 views

Convergence of k-means or EM on Mixture of Gaussians

There are many algorithms for learning mixture of Gaussians but typically k-means/EM is used in practice. My question is related to the performance of k-means/EM for MoG. Recently, I came across this ...
3
votes
0answers
835 views

Fit Gaussian Mixture model directly to the mixture density

The core of the question is: Can I estimate the parameters of a gaussian mixture model (with EM or Dirichlet Process) from a mixture density directly, that is, without using data drawn from such ...
3
votes
0answers
483 views

How to calculate BIC for multidimensional problem

Sorry for this question, but I am really not sure how to calculate BIC for my situation. My models are mixtures of normals with different number of components. Variances are equal for all components ...
3
votes
0answers
379 views

Diffusion coefficient from double-normal probability density function

The spread of individuals of species is often described by so-called dispersal kernels. The main parameter of spread is then often the variance defined as the average squared movement distance of a ...
3
votes
0answers
228 views

Why isn't a gaussian mixture prone to overfitting?

Consider a Gaussian mixture of 2 components and a dataset of size $N$. The EM algorithm use the data to estimate: the model parameters: the means $\mu_1, \mu_2$ (say the covariances matrices are ...
3
votes
0answers
429 views

A better bayesian way of modelling autoregressive mixtures

I have a JAGS hierarchical model which includes a temporal sub-model for the primary vote share between four party groups (LNP, Labor, Green, and Other). For each day in the temporal model, the vote ...
2
votes
0answers
35 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
2
votes
0answers
25 views

In a normal mixture model, what general happens if the number of groups to be summed far exceeds the dimensions of each normal?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_kI_d) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean vector $\mu$ and d-...
2
votes
0answers
35 views

Multimodality of mixtures of more than two Normal distributions

Let $$\phi(x;\mu,\sigma) = \frac{1}{\sigma \sqrt{2\pi}} \exp \left(- \frac{(x-\mu)^2}{2\sigma^2}\right)$$ denote the Gaussian density function ($\sigma > 0$). Let $$f(x) = \sum_{i=1}^N p_i \...
2
votes
1answer
346 views

GMM EM algorithm complexity per iteration

I was fitting GMM clusters with diagonal covariance on my data using EM with $n$ (=5e6) points, each having $m$ (=160) ...
2
votes
0answers
71 views

Gaussian Mixture Division

In the study of probabilistic graphical models (PGMs), the loopy belief update propagation (LBUP) message passing algorithm requires the division of unnormalised probability distributions. If the ...
2
votes
0answers
64 views

Distribution of the initializing set at K-means++

There is a well-known modification of the initializing step of K-means, named K-means++. It chooses cluster centers with probability proportional to its squared distance from the point's closest ...
2
votes
0answers
396 views

Approximating Uniform Distribution with Mixture of Gaussians

Let $T$ be a compact, connected, proper subset of $\mathbb{R}^3:\quad T \subset \mathbb{R}^3$. Further let $\left\{ \boldsymbol{\mu}_i \right\}_{i=1}^n$ be a given finite set of $n$ point in $T$: $$ \...
2
votes
0answers
540 views

Programming a mixture of a Gamma with a Normal distribution using R

I have some data x in R which seems to be a mixture of a Gamma and Normal distribution. Therefore I'd like to model this as a mixture model consisting of said distributions, but I don't know how to ...
2
votes
1answer
222 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
2
votes
0answers
53 views

Variance of mix of normals

Suppose we have $n$ random variables distributed normally with the same mean and different variances. Suppose we know these variances. Which will be the variance of the marginal distribution induced ...
2
votes
0answers
416 views

Can a Gaussian mixture model be specified using a regression equation?

From: https://stats.stackexchange.com/a/236297/22199, I quote A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A ...
2
votes
0answers
150 views

Which cluster analysis for ordinal temporal data?

I would like to perform a cluster analysis but I’m not sure which is the best algorithm to apply to my data. My dataset is made of 200 cases (but the sample size can be enlarged). For each case, I ...
2
votes
0answers
873 views

Entropy of a set of categorical variables

In the context of Expectation-Maximization, I would like to compute te entropy factor in order to get the value of the lower bound when the algorithm converged. This lower bound can be expressed as: ...
2
votes
0answers
114 views

Expectation Maximization Gaussian Mixture Example

I am a biologist trying to understand expectation maximization for a mixture of two Gaussian distributions. I think I understand how to deal with the means of the two distributions, but I don't know ...
2
votes
0answers
211 views

Confusion with EM Algorithm for Gaussian Mixture?

I am trying to learn EM Algorithm for Gaussian Mixture. But not able to understand few stuffs. This is what I have understood. Consider GMM with k components. $$ p( \mathbf{x}| \mathbf{\alpha_{k}},\...
2
votes
0answers
163 views

Particle Filter and Gaussian Mixture

Let an observation model be given as $f(y_t|x_t)$ - this pdf is assumed to be nontrivial (not normal, not linear). The observation model is assumed to be known. Despite there is a state evolution ...
2
votes
0answers
268 views

Calculate number of standard deviations separating two multivariate Gaussians?

Given a set of multivariate Gaussian distributions (from fitting a Gaussian mixture model) I would like to be able to calculate the likelihood that a data point drawn from one Gaussian will improperly ...
2
votes
1answer
381 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian (...
2
votes
0answers
95 views

How can I measure separability between different number of instance of one feature vector?

How can I measure separability between different number of instance of one feature vector? For example the main vector is V=[1 1 2 3 4 5 7 8 10 100 1000 99 999 54] ...
2
votes
0answers
442 views

Should I add noise to my truth data before before training my classifier?

My task is to develop a system that will take in a series of measurements and return the probability that an object is a type 1, type 2,... type n. I will refer to the system I have to create as a ...
2
votes
1answer
812 views

Estimate Gaussian (mixture) density from a set of weighted samples

I am trying to find if there exists a way to find the spatial distribution over a set of points where the points are weighted. If I have "n" points in the (x,y) space, I can fit a mixture of Gaussians ...
1
vote
0answers
12 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
1
vote
0answers
11 views

How do I properly scale the covariance matrix in a weighted Gaussian mixture model for new samples?

I am trying to implement the method for computing a Gaussian mixture model from samples with known weights as detailed in section III of: EM Algorithms for Weighted-Data Clustering with Application ...
1
vote
0answers
42 views

skew G-Jensen-Shannon divergence between multivariate gaussian calculation discrepancy

I'm trying to calculate the Jensen-Shannon divergence between two multivariate Gaussians. I found a closed-form expression both for the KL divergence and JS divergence between two Gaussians in this ...
1
vote
0answers
47 views

Hidden Markov Model for classification

I have fitted a Gaussian mixture model to my data. This Gaussian mixture model is the combination of two Gaussian distributions. I call the first Gaussian distribution state 1 and the second Gaussian ...
1
vote
1answer
163 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
1
vote
0answers
129 views

Stopping criteria for gaussian mixture models

As I can read from the source code of scikit-learn, the stopping criteria for the iterative algorithm of Expectation Maximization (in my case applied to fitting Gaussian mixture models) is to put a ...
1
vote
0answers
15 views

Does mixture of sigmoids make sense given the theories about mixture of bernoullis?

Mixture of bernoullis is the combination of bernoulli distributions, which can be illustrated by the sampling process of K bags of D coins, here is a quick tutorial about it https://cedar.buffalo.edu/~...
1
vote
0answers
34 views

Determining number of components in mixtures of normal distributions with common mean

This is a pretty simple question, suppose we want to fit a mixture distribution of multivariate normals with common mean $$y_i \sim \sum_k \pi_k N(\mu, \Sigma_k)$$ What is the preferred approach for ...
1
vote
0answers
28 views

Is there any background for constraining covariances on fitting GMM?

On clustering data using GMM model, I often see the option to constrain covariances of each clustered GMM. For example, http://scikit-learn.org/0.16/auto_examples/mixture/plot_gmm_classifier.html ...
1
vote
0answers
1k views

Gaussian Mixture Model with labels in Python

I have data X and corresponding labels y and want to fit a Gaussian Mixture model to it. In Matlab, one has the option of specifying initial labels. I am trying to do the same in Python. This is what ...
1
vote
0answers
85 views

Conditional sampling from a multivariate Gaussian Mixture

I am using scikit-learn to fit a gaussian mixture on a non-parametric multivariate distribution with three variables $ \mathbf{X} = (X_1, X_2, X_3) $ I want to sample from that distribution given ...
1
vote
0answers
143 views

Entropy of a mixture of Gaussians

I need to estimate as fast and accurately as possible the differential entropy of a mixture of $K$ multivariate Gaussians: $$ \mathcal{H}[q] = -\sum_{k=1}^K w_k \int q_k(\textbf{x}) \log \left[\sum_{...
1
vote
0answers
32 views

Can the hidden states of a HMM be interpreted as number of clusters underlying the data?

Trying to understand the physical significance of the number of hidden states of a HMM. Should they be interpreted as number of clusters in the data? If not, why? Or they should be interpreted as the ...
1
vote
0answers
29 views

MIxture model in R to generate noise in data

I have a bit of code in R that adds noise to an harmonic series according to a normal distribution: ...