Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

0
votes
1answer
36 views

Inverse distribution of gaussian mixture

In one of the papers I've encountered, the authors propose a copula function $$ c(u_1, \ldots, u_d; \Theta) = \frac{\psi(y_1, \ldots, y_d; \Theta)}{\prod_{j=1}^{d}\psi_j (y_j)}$$ where $\psi(y_1, \...
0
votes
0answers
13 views

Do finite mixtures of normals have heavier tails than normal? [on hold]

Consider a finite mixture of normals $$f(x;\mu_1,\dots,\mu_n, \sigma_1,\dots,\sigma_n) = \sum_{i=1}^n \omega_i \phi(x;\mu_i,\sigma_i),$$ where $\sum_{i=1}^n \omega_i = 1$ and $\phi(x;\mu_i,\sigma_i)$...
0
votes
0answers
34 views

Interpretation of Mclust Gaussian Mixture Model [on hold]

I'm doing a study on a type of protein in the brain. In theory, everybody has some level of this protein, but a subgroup might have more of it. I'm using a Gaussian mixture model to try to find a cut-...
0
votes
0answers
6 views

Biased viterbi training result

I try to use GMM-HMM model to infer the topic of sentences in a short paragraph. While instead of using normal Baum-Welch optimization, I use viterbi training as follows. I use average of word ...
-1
votes
0answers
22 views

Gaussian mixture model (GMM) to cluster high dimensional dataset

My data set has ~20.000 dimensions. I want to use "Gaussian mixture model " to cluster them. To Construct mixtures, multivariate distributions are required. Given that high dimensionality, how ...
0
votes
0answers
15 views

Expectation of normal RV conditional on normal mixture

Let $v\sim\text{Normal}\left(\mu,\sigma_v^2\right)$ a random variable with $\mu>0$ and $u\sim\text{Normal}\left(0,\sigma_u^2\right)$ Let $k\sim\text{Binomial}\left(N,p\right)$ a random variable ...
1
vote
1answer
40 views

Can we use a mixture of normal distributions while optimising likelihood?

Let's assume that we generate some values by a mixture of two Gaussians. Now we want to find the parameters of the two Gaussians by likelihood maximisation. One good expect that the optimisation will ...
0
votes
0answers
7 views

Interpretation of NLP pipeline for topic discovery using gaussian mixture model clustering

I built a pipeline that does the following to discover topics out of a (very big: 50k docs per ~350 terms) Term Document Matrix: Compute the TfIdf score for each Term x Document pair; Rescale each ...
0
votes
1answer
33 views

Calculation of Bayeain rule as classifier for mixture Gaussian model

Here is a paper which used a bayesian classification based on Gaussian mixture model I read many article saying that we can fit a gaussian mixture model to a data and then, based on the estimated ...
1
vote
0answers
27 views

skew G-Jensen-Shannon divergence between multivariate gaussian calculation discrepancy

I'm trying to calculate the Jensen-Shannon divergence between two multivariate Gaussians. I found a closed-form expression both for the KL divergence and JS divergence between two Gaussians in this ...
0
votes
0answers
15 views

How to forecast a time-series with a dynamic time unit?

I'm working on a forecasting problem, and I'm not sure if the data requires any transformation because the unit of time is dynamic. I'm working with an education data set where I have data on ...
0
votes
0answers
14 views

How to 'normalize' the product of two variables from Gaussian distribution?

I have two variables, x1 and x2, which are sampled from two Gaussian distributions respectively. I created an iteraction term x3 which is x1 multiply x2. Not surprisingly, x3 has very fat tail, ie., ...
2
votes
1answer
77 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
4
votes
2answers
119 views

Why do we use Gaussian distributions in Variational Autoencoder?

I still don't understand why we force the distribution of the hidden representation of a Variational Autoencoder (VAE) to follow a multivariate normal distribution. Why this specific distribution and ...
1
vote
0answers
39 views

Hidden Markov Model for classification

I have fitted a Gaussian mixture model to my data. This Gaussian mixture model is the combination of two Gaussian distributions. I call the first Gaussian distribution state 1 and the second Gaussian ...
0
votes
1answer
16 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
1
vote
1answer
250 views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
3
votes
0answers
26 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
1
vote
1answer
89 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
2
votes
1answer
148 views

Sampling from Gaussian mixture models, when are the sampled data independent?

Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions $p(x) = \sum\limits_{i = 1}^N w_i \mathcal{N}(x;\mu_i, \Sigma_i)$ where $w_i$ are the weights. Now I sample some points $\...
7
votes
2answers
720 views

Why use a Gaussian mixture model?

I am learning about Gaussian mixture models (GMM) but I am confused as to why anyone should ever use this algorithm. How is this algorithm better than other standard clustering algorithm such as $K$-...
0
votes
0answers
10 views

Imposing constraints on observation model in a HMM

I have $N$ observations ($x_1, x_2,.. ,x_N$) from a HMM with $K$ latent states. The M step for computing the observation model $\mu_k$ involves maximizing the expression: $$ L = \sum_{n=1}^{N}{ln \...
1
vote
1answer
14 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
0
votes
0answers
9 views

Can we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?

I learned from this answer that: A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A gaussian-mixture is the ...
0
votes
0answers
19 views

Redefining latent variables as observed data

This was just a thought that occurred to me, but technically, is it possible to redefine what I treat as latent variables and what I treat as data? For example, lets assume I have a set of latent ...
0
votes
0answers
34 views

Standard deviation in multimodal data

I have a dataset, 90% of observations are unimodal normal (with couple of outliers per feature), 10% are mixture of normals, components have the same standard deviations. Data contains outliers => ...
1
vote
1answer
27 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
0
votes
0answers
14 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
1
vote
0answers
64 views

Stopping criteria for gaussian mixture models

As I can read from the source code of scikit-learn, the stopping criteria for the iterative algorithm of Expectation Maximization (in my case applied to fitting Gaussian mixture models) is to put a ...
0
votes
0answers
25 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
2
votes
2answers
167 views

Compute quantile function from a mixture of Normal distribution

I have this mixture of normal distribution: $$X \sim \frac{1}{2}\mathcal{N}(\mu_{x_1}=10,\,\sigma_{x_1}^{2}=1)+\frac{1}{2}\mathcal{N}(\mu_{x_2}=13,\,\sigma_{x_2}^{2}=1)$$ How can i compute the ...
1
vote
1answer
39 views

Calculating the probability that an observation comes from either population A or B

If I have two normal distributions A (mean = 0, variance = 4) and B (mean = 0, variance = 16), how can I calculate the probability that an observation with magnitude 2 comes from A?
2
votes
1answer
38 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
1
vote
0answers
21 views

Tensorflow InvalidArgumentError: The determinant is not finite [closed]

I'm trying to fit a Mixture of Gaussians to a data set. First the data is clustered using K-Means Clustering. Each cluster is then fitted with a Gaussian.To avoid inversion of large covariance matrix, ...
2
votes
1answer
38 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
107 views

Gaussian Mixture: is this plot right?

I'm studying about Gaussian Mixtures and I decided to play around with it in Python, but I'm not entirely sure if I understand it fully. I generated some data, and then calculated the Gaussian ...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
0
votes
1answer
112 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
2
votes
0answers
32 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
0
votes
0answers
21 views

GMM model of the joint distribution from multivariate marginals

I have two multivariate Gaussian variables $X \sim \mathcal{N}(\boldsymbol {\mu}_X \in \mathcal R^d,\boldsymbol {\Sigma}_X \in \mathcal R^{d \times d})$ and $Y \sim \mathcal{N}(\boldsymbol {\mu}_Y \...
0
votes
1answer
121 views

Gaussian Mixture Model with k=n clusters

Suppose that we construct a Gaussian Mixture Model with FIXED COVARIANCE on $n$ points using $k=n$ clusters. Is it the case that the Maximum Likelihood parameters put each of the $n$ points in their ...
1
vote
1answer
94 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
2
votes
1answer
230 views

Scikit-Learn Gaussian Mixture: How can log-probabilities be positive? [closed]

I am fitting a Gaussian Mixture model: gm = GaussianMixture(n_components=K) gm.fit(features) When I do: ...
0
votes
1answer
65 views

Is there anything like “implied density” in this experiment?

A basket of balls is dropped into a maze. When a ball is dropped into the maze at the top it moves downward, pulled by gravity, through a series of nails. The ball then falls down to a new level where ...
1
vote
1answer
93 views

Reducibility between Gaussian Mixture Models and Gaussian Processes

I am studying gaussian processes and I have already discrete amount of knowledge in gaussian mixture models. I am here to undersrtand if with a gaussian process you can fit a gaussian mixture model. ...
1
vote
1answer
63 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
1
vote
1answer
90 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
0
votes
1answer
29 views

Concise way to visualize / compare many Gaussian mixtures

I have 5,000 samples drawn from each of approximately 50,000 distributions. I have good reason to expect most of them to be normally distributed, and I expect some of them to be multi-modal (mixture ...
0
votes
0answers
20 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
252 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...