Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

0
votes
0answers
19 views

Calculating the CDF value of a random variable [closed]

I have the following equation: It seems that there is a random variable inside the CDF brackets of a normal distribution in the second half of the equation ($z_1$). It seems that we can use ...
0
votes
0answers
10 views

Standard deviation in multimodal data

I have a dataset, 90% of observations are unimodal normal (with couple of outliers per feature), 10% are mixture of normals, components have the same standard deviations. Data contains outliers => ...
1
vote
1answer
25 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
0
votes
0answers
11 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
1
vote
0answers
14 views

Stopping criteria for gaussian mixture models

As I can read from the source code of scikit-learn, the stopping criteria for the iterative algorithm of Expectation Maximization (in my case applied to fitting Gaussian mixture models) is to put a ...
0
votes
0answers
20 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
2
votes
2answers
50 views

Compute quantile function from a mixture of Normal distribution

I have this mixture of normal distribution: $$X \sim \frac{1}{2}\mathcal{N}(\mu_{x_1}=10,\,\sigma_{x_1}^{2}=1)+\frac{1}{2}\mathcal{N}(\mu_{x_2}=13,\,\sigma_{x_2}^{2}=1)$$ How can i compute the ...
1
vote
1answer
34 views

Calculating the probability that an observation comes from either population A or B

If I have two normal distributions A (mean = 0, variance = 4) and B (mean = 0, variance = 16), how can I calculate the probability that an observation with magnitude 2 comes from A?
2
votes
1answer
33 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
1
vote
0answers
7 views

Tensorflow InvalidArgumentError: The determinant is not finite [closed]

I'm trying to fit a Mixture of Gaussians to a data set. First the data is clustered using K-Means Clustering. Each cluster is then fitted with a Gaussian.To avoid inversion of large covariance matrix, ...
2
votes
1answer
31 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
87 views

Gaussian Mixture: is this plot right?

I'm studying about Gaussian Mixtures and I decided to play around with it in Python, but I'm not entirely sure if I understand it fully. I generated some data, and then calculated the Gaussian ...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
0
votes
1answer
23 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
2
votes
0answers
27 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
0
votes
0answers
20 views

GMM model of the joint distribution from multivariate marginals

I have two multivariate Gaussian variables $X \sim \mathcal{N}(\boldsymbol {\mu}_X \in \mathcal R^d,\boldsymbol {\Sigma}_X \in \mathcal R^{d \times d})$ and $Y \sim \mathcal{N}(\boldsymbol {\mu}_Y \...
0
votes
1answer
29 views

Gaussian Mixture Model with k=n clusters

Suppose that we construct a Gaussian Mixture Model with FIXED COVARIANCE on $n$ points using $k=n$ clusters. Is it the case that the Maximum Likelihood parameters put each of the $n$ points in their ...
1
vote
1answer
69 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
2
votes
1answer
125 views

Scikit-Learn Gaussian Mixture: How can log-probabilities be positive? [closed]

I am fitting a Gaussian Mixture model: gm = GaussianMixture(n_components=K) gm.fit(features) When I do: ...
0
votes
1answer
64 views

Is there anything like “implied density” in this experiment?

A basket of balls is dropped into a maze. When a ball is dropped into the maze at the top it moves downward, pulled by gravity, through a series of nails. The ball then falls down to a new level where ...
1
vote
1answer
26 views

Reducibility between Gaussian Mixture Models and Gaussian Processes

I am studying gaussian processes and I have already discrete amount of knowledge in gaussian mixture models. I am here to undersrtand if with a gaussian process you can fit a gaussian mixture model. ...
1
vote
1answer
24 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
1
vote
1answer
46 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
0
votes
1answer
24 views

Concise way to visualize / compare many Gaussian mixtures

I have 5,000 samples drawn from each of approximately 50,000 distributions. I have good reason to expect most of them to be normally distributed, and I expect some of them to be multi-modal (mixture ...
0
votes
0answers
13 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
174 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
3
votes
1answer
73 views

How to cluster parts of broken line made of points?

I am studying clustering techniques and i am pretty new at this topic. Here is my problem: I created a 5 lines which are made of points. This lines are supposed to be continuous and they look like ...
3
votes
2answers
238 views

Gaussian mixture model - does an improper uniform prior give a proper posterior?

We draw $n$ i.i.d. points $x_1 , x_2 , ..., x_n$ from the following Gaussian mixture: $$p(x|\mu_1,\mu_2) = \frac{1}{2} \text{N} (x|\mu_1,1) + \frac{1}{2} \text{N} (x|\mu_2,1).$$ The prior is the ...
0
votes
1answer
46 views

Are Gaussian Mixture Models stochastic or deterministic?

Each time we generate a gmm model, we obtain slightly different clusters. Can we hence say gmm is stochastic? We obtain the same clusters if a random seed is set; does this mean given a random seed, ...
1
vote
0answers
12 views

Does mixture of sigmoids make sense given the theories about mixture of bernoullis?

Mixture of bernoullis is the combination of bernoulli distributions, which can be illustrated by the sampling process of K bags of D coins, here is a quick tutorial about it https://cedar.buffalo.edu/~...
1
vote
0answers
28 views

Determining number of components in mixtures of normal distributions with common mean

This is a pretty simple question, suppose we want to fit a mixture distribution of multivariate normals with common mean $$y_i \sim \sum_k \pi_k N(\mu, \Sigma_k)$$ What is the preferred approach for ...
1
vote
1answer
51 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
3
votes
0answers
30 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
0
votes
0answers
5 views

Guassian Mixture Classification: interpretating component x variable means matrix

The Guassian Mixture model output by mclust::Mclust() function has a $parameters$mean element which is a matrix with dimensions ...
1
vote
0answers
18 views

Is there any background for constraining covariances on fitting GMM?

On clustering data using GMM model, I often see the option to constrain covariances of each clustered GMM. For example, http://scikit-learn.org/0.16/auto_examples/mixture/plot_gmm_classifier.html ...
0
votes
0answers
43 views

Output Size of Mixture Density Networks

I am working on a neural network in which the final output layer will be a Mixture Density Network (MDN), but am confused about the shape of the values that final layer should return. In the paper in ...
2
votes
1answer
38 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
1
vote
2answers
289 views

Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?

I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like: Since this was ...
0
votes
0answers
22 views

Combining GMM's in different dimensions

I have input $X$, which follows a distribution $P(X)$, which is best modeled using a mixture of Gaussians. I also have another random variable $T$, which is also best modeled by a mixture of Gaussians....
0
votes
1answer
130 views

Is it important to make a feature scaling before using Gaussian Mixture Model?

Is it important to make a feature scaling before using Gaussian Mixture Model? and why is it important while we are using probability in getting our clusters's parameters (mean and covariance matrix). ...
1
vote
0answers
396 views

Gaussian Mixture Model with labels in Python

I have data X and corresponding labels y and want to fit a Gaussian Mixture model to it. In Matlab, one has the option of specifying initial labels. I am trying to do the same in Python. This is what ...
2
votes
1answer
65 views

Correlated random variables from mixture distributions

Let I have three random variables whose density is a mixture of two Normals with these parameters: $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$ $\...
4
votes
2answers
395 views

Is a GMM-HMM equivalent to a no-mixture HMM enriched with more states?

I'm trying to model sequence data that has 5 hidden states. Observation data conditional to each state is gaussian except for one state for which mixture of 2 gaussians seems more appropriate. ...
2
votes
1answer
551 views

A Gaussian Mixture Model Is a Universal Approximator of Densities

When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following: A Gaussian mixture model is a universal approximator of densities, in the ...
0
votes
1answer
41 views

Decompose/split a single multivariate gauss into random gaussian mixture

Say, there is a single $n$-dimensional multivariate Gaussian. $$Gauss_a(\mu_a,\Sigma_a) $$ $\mu_a$ is $1\times n$ vector and $\Sigma_a$ is $n\times n$ matrix. Is there any easy way to decompose/...
0
votes
1answer
41 views

Why do we assume Gaussian margins in Gaussian mixture models [duplicate]

A Gaussian mixture model is a weighted sum of Gaussian densities, i.e., $L(\theta) = \sum_{i=1}^{m} \pi_{i} f(x_i)$ where $m$ is the number of the mixture component. Hence, Gaussian mixture ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
1
vote
0answers
52 views

Conditional sampling from a multivariate Gaussian Mixture

I am using scikit-learn to fit a gaussian mixture on a non-parametric multivariate distribution with three variables $ \mathbf{X} = (X_1, X_2, X_3) $ I want to sample from that distribution given ...
1
vote
0answers
59 views

Entropy of a mixture of Gaussians

I need to estimate as fast and accurately as possible the differential entropy of a mixture of $K$ multivariate Gaussians: $$ \mathcal{H}[q] = -\sum_{k=1}^K w_k \int q_k(\textbf{x}) \log \left[\sum_{...
1
vote
0answers
29 views

Can the hidden states of a HMM be interpreted as number of clusters underlying the data?

Trying to understand the physical significance of the number of hidden states of a HMM. Should they be interpreted as number of clusters in the data? If not, why? Or they should be interpreted as the ...