Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
4
votes
0answers
28 views

Implementation of EM algorithm confusion

Here EM algorithm manually implemented, there's a question of the implementation in R of the EM algorithm for 2 mixed gaussians. The answer has a supposedly correct implementation. However, don't the ...
1
vote
0answers
7 views

How do I properly scale the covariance matrix in a weighted Gaussian mixture model for new samples?

I am trying to implement the method for computing a Gaussian mixture model from samples with known weights as detailed in section III of: EM Algorithms for Weighted-Data Clustering with Application ...
0
votes
0answers
6 views

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture Problem description I am trying to normalize multivariate time series data. The ...
0
votes
1answer
14 views

Expected Misclustering rate

I am reading this paper on minimax clustering error rates on high-dimensional Gaussian mixtures. The authors define a metric for expected misclustering rate as follows: For a two-component ...
0
votes
0answers
23 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
0
votes
0answers
25 views

Confidence regions after fitting a 2 parameter gaussian mixture model?

Suppose I have a gaussian mixture model with 2 parameters $(u,v)$ and 2 parts. The model is $P({x_i}|u,v)=uN(x_i|\mu_1^{i} = x_i^2/v,\sigma_1^{i}) + (1-u)N(x_i|\mu_2^{i} = x_i^3/2v^2,\sigma_2^i)$. ...
4
votes
1answer
56 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
0
votes
0answers
15 views

Training Hidden Markov model with GMM, nan appears after some iterations

Problem During the training process of my continuous observation sequence data using HMM with GMM mixtures, the cost function reduces gradually and it becomes NaN ...
0
votes
0answers
37 views

Maximum likelihood convergence in mixture gaussian

Suppose there are two datasets $D_{1}$ and $D_{2}$ with same structure, which means the cluster and cluster proportion is the same. The only difference between them is that the size of $D_{1}$ is $n_{...
0
votes
0answers
12 views

How is jaccard similarity used to find the similarity between Bootstrap samples when measuring stability of EM?

Im reading the answer on "how to determine number of clusters in EM algorithm". How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results? One of the ...
0
votes
1answer
46 views

Inverse distribution of gaussian mixture

In one of the papers I've encountered, the authors propose a copula function $$ c(u_1, \ldots, u_d; \Theta) = \frac{\psi(y_1, \ldots, y_d; \Theta)}{\prod_{j=1}^{d}\psi_j (y_j)}$$ where $\psi(y_1, \...
0
votes
0answers
8 views

Biased viterbi training result

I try to use GMM-HMM model to infer the topic of sentences in a short paragraph. While instead of using normal Baum-Welch optimization, I use viterbi training as follows. I use average of word ...
0
votes
0answers
17 views

Expectation of normal RV conditional on normal mixture

Let $v\sim\text{Normal}\left(\mu,\sigma_v^2\right)$ a random variable with $\mu>0$ and $u\sim\text{Normal}\left(0,\sigma_u^2\right)$ Let $k\sim\text{Binomial}\left(N,p\right)$ a random variable ...
1
vote
1answer
44 views

Can we use a mixture of normal distributions while optimising likelihood?

Let's assume that we generate some values by a mixture of two Gaussians. Now we want to find the parameters of the two Gaussians by likelihood maximisation. One good expect that the optimisation will ...
0
votes
0answers
11 views

Interpretation of NLP pipeline for topic discovery using gaussian mixture model clustering

I built a pipeline that does the following to discover topics out of a (very big: 50k docs per ~350 terms) Term Document Matrix: Compute the TfIdf score for each Term x Document pair; Rescale each ...
0
votes
1answer
34 views

Calculation of Bayeain rule as classifier for mixture Gaussian model

Here is a paper which used a bayesian classification based on Gaussian mixture model I read many article saying that we can fit a gaussian mixture model to a data and then, based on the estimated ...
1
vote
0answers
31 views

skew G-Jensen-Shannon divergence between multivariate gaussian calculation discrepancy

I'm trying to calculate the Jensen-Shannon divergence between two multivariate Gaussians. I found a closed-form expression both for the KL divergence and JS divergence between two Gaussians in this ...
0
votes
0answers
16 views

How to forecast a time-series with a dynamic time unit?

I'm working on a forecasting problem, and I'm not sure if the data requires any transformation because the unit of time is dynamic. I'm working with an education data set where I have data on ...
0
votes
0answers
15 views

How to 'normalize' the product of two variables from Gaussian distribution?

I have two variables, x1 and x2, which are sampled from two Gaussian distributions respectively. I created an iteraction term x3 which is x1 multiply x2. Not surprisingly, x3 has very fat tail, ie., ...
2
votes
1answer
92 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
4
votes
2answers
312 views

Why do we use Gaussian distributions in Variational Autoencoder?

I still don't understand why we force the distribution of the hidden representation of a Variational Autoencoder (VAE) to follow a multivariate normal distribution. Why this specific distribution and ...
1
vote
0answers
45 views

Hidden Markov Model for classification

I have fitted a Gaussian mixture model to my data. This Gaussian mixture model is the combination of two Gaussian distributions. I call the first Gaussian distribution state 1 and the second Gaussian ...
0
votes
1answer
17 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
1
vote
1answer
507 views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
3
votes
0answers
33 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
1
vote
1answer
128 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
2
votes
1answer
170 views

Sampling from Gaussian mixture models, when are the sampled data independent?

Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions $p(x) = \sum\limits_{i = 1}^N w_i \mathcal{N}(x;\mu_i, \Sigma_i)$ where $w_i$ are the weights. Now I sample some points $\...
7
votes
2answers
784 views

Why use a Gaussian mixture model?

I am learning about Gaussian mixture models (GMM) but I am confused as to why anyone should ever use this algorithm. How is this algorithm better than other standard clustering algorithm such as $K$-...
0
votes
0answers
13 views

Imposing constraints on observation model in a HMM

I have $N$ observations ($x_1, x_2,.. ,x_N$) from a HMM with $K$ latent states. The M step for computing the observation model $\mu_k$ involves maximizing the expression: $$ L = \sum_{n=1}^{N}{ln \...
1
vote
1answer
15 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
0
votes
0answers
10 views

Can we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?

I learned from this answer that: A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A gaussian-mixture is the ...
0
votes
0answers
19 views

Redefining latent variables as observed data

This was just a thought that occurred to me, but technically, is it possible to redefine what I treat as latent variables and what I treat as data? For example, lets assume I have a set of latent ...
0
votes
0answers
49 views

Standard deviation in multimodal data

I have a dataset, 90% of observations are unimodal normal (with couple of outliers per feature), 10% are mixture of normals, components have the same standard deviations. Data contains outliers => ...
2
votes
1answer
28 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
0
votes
0answers
15 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
1
vote
0answers
110 views

Stopping criteria for gaussian mixture models

As I can read from the source code of scikit-learn, the stopping criteria for the iterative algorithm of Expectation Maximization (in my case applied to fitting Gaussian mixture models) is to put a ...
0
votes
0answers
26 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
2
votes
2answers
220 views

Compute quantile function from a mixture of Normal distribution

I have this mixture of normal distribution: $$X \sim \frac{1}{2}\mathcal{N}(\mu_{x_1}=10,\,\sigma_{x_1}^{2}=1)+\frac{1}{2}\mathcal{N}(\mu_{x_2}=13,\,\sigma_{x_2}^{2}=1)$$ How can i compute the ...
1
vote
1answer
40 views

Calculating the probability that an observation comes from either population A or B

If I have two normal distributions A (mean = 0, variance = 4) and B (mean = 0, variance = 16), how can I calculate the probability that an observation with magnitude 2 comes from A?
2
votes
1answer
39 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
1
vote
0answers
26 views

Tensorflow InvalidArgumentError: The determinant is not finite [closed]

I'm trying to fit a Mixture of Gaussians to a data set. First the data is clustered using K-Means Clustering. Each cluster is then fitted with a Gaussian.To avoid inversion of large covariance matrix, ...
2
votes
1answer
48 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
138 views

Gaussian Mixture: is this plot right?

I'm studying about Gaussian Mixtures and I decided to play around with it in Python, but I'm not entirely sure if I understand it fully. I generated some data, and then calculated the Gaussian ...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
0
votes
1answer
147 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
2
votes
0answers
33 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
0
votes
0answers
24 views

GMM model of the joint distribution from multivariate marginals

I have two multivariate Gaussian variables $X \sim \mathcal{N}(\boldsymbol {\mu}_X \in \mathcal R^d,\boldsymbol {\Sigma}_X \in \mathcal R^{d \times d})$ and $Y \sim \mathcal{N}(\boldsymbol {\mu}_Y \...
0
votes
1answer
159 views

Gaussian Mixture Model with k=n clusters

Suppose that we construct a Gaussian Mixture Model with FIXED COVARIANCE on $n$ points using $k=n$ clusters. Is it the case that the Maximum Likelihood parameters put each of the $n$ points in their ...
1
vote
1answer
111 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
2
votes
1answer
290 views

Scikit-Learn Gaussian Mixture: How can log-probabilities be positive? [closed]

I am fitting a Gaussian Mixture model: gm = GaussianMixture(n_components=K) gm.fit(features) When I do: ...