Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
1
vote
0answers
9 views

How to fit mixture of gaussians with identical mean?

Say I have data generated by a mixture of gaussians whose components have the same mean, but very different covariances, like the one generated by this code: ...
1
vote
1answer
16 views

What does the y parameter in .fit() of scikit-learn's Gaussian Mixture Model do?

From my understanding, Gaussian Mixture Models are an unsupervised method and can perform clustering similar to k-means. In the scitkit-learn implementation of GMM, the .fit() function (and ....
2
votes
1answer
40 views

Conditional Probability - Mixture Model

I know that the likelihood in a p-dimensional Gaussian mixture model is given by $$ p(s|\theta) = \sum_{b_1 = 0}^1\cdots\sum_{b_p = 0}^1\left[ \prod_{i=1}^pw^{1-b_i}(1-w)^{1-b_i}\right]\phi_p(s|\mu(b,...
0
votes
0answers
9 views

Probabilistic Models, what do they infer?

As per my understanding, Mixture Models such as GMM, Probabilistic Models such as Variation Autoencoder, they explain the latent space behind the features. But how they turn from learning latent space ...
0
votes
1answer
24 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
2
votes
1answer
43 views

Distribution of Distributions | percent point function

I have the following normal distribution (Primary Distribution): Mean = 7 Sigma = 0.5 and a list of other normal distributions (Secondary Distributions): (python list of tupels, each tupel ...
2
votes
1answer
65 views

Figueiredo and Jain's Gaussian mixture EM convergence criterion

I have implemented and been playing around Figueiredo & Jain 's trainer in this paper http://www.lx.it.pt/~mtf/IEEE_TPAMI_2002.pdf for Gaussian mixture. Fig. 2 in the paper depicts the full ...
0
votes
0answers
37 views

Gibbs Sampler for GMM

In Rasmussen's paper it is introduced a Gibbs sampler to make inference about a standard Gaussian Mixture Model. To simplify, assume the 1-d case with basic hierarchical structure, that is: $x_i|...
0
votes
2answers
28 views

Convergence of EM algorithm

I am aware that EM eventually converges. However, I still have some confusions regarding this property: 1: As far as I am aware, HMM, Gaussian mixture model and MCMC can converge and all of them use ...
0
votes
0answers
23 views

what is wrong with my Gaussian Mixture density estimation fitting (Python)?

I have a data set (1D) link: dataset, which has values ranging from 21,000 to 8,000,000. When i plot histogram of the log values, i can see there are two peaks, roughly. I tried to fit Gaussian ...
1
vote
0answers
24 views

Closed form for Finite Gaussian Mixture Model when weights are known and prior variance can be 0

Suppose I have a normal likelihood $x|\theta \sim N(\theta, \sigma^2_{known})$ where the variance is known and a mixture prior $\theta \sim p * N(\mu_1, \sigma^2_1) + (1-p) * N(\mu_2, \sigma^2_2)$, ...
0
votes
2answers
31 views

What is the interpretation of the weights in the GMM?

GMM is $p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the interpretation of the weights here? Do ...
1
vote
0answers
18 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
0
votes
0answers
7 views

How to deduct the complete likelihood of mixture normal in EM algorithm

We have the well known complete likelihood of mixture normal in EM algorithm: Here $Z$ is a random variable that it has probability $\pi_k$ to choose k-th normal variable $X_k:N(\mu_k,\sigma_k).$ We ...
0
votes
2answers
78 views

Suggest a model for this dataset

I have a time series data set (the Old Faithful geyser data available here: http://www.gatsby.ucl.ac.uk/teaching/courses/ml1-2012/geyser.txt). Plotting the eruption duration on the x axis and the ...
2
votes
2answers
46 views

Interpreting mixture of Gaussians (Variational Inference)

I've recently stated reading about mixture models and variational inference in this excellent paper, but I'm having troubles dissecting the models described, and have a couple of questions. Please see ...
2
votes
2answers
32 views

Can Gaussian Mixture Model Clustering tell me something about the distribution of my data?

I have 10,000 vectors originating from 5 separate classes (2,000 each). I use Gaussian Mixture Model clustering (in Python) to cluster the 10,000 vectors, telling the algorithm to cluster the data ...
1
vote
2answers
20 views

Why is Matlab's cluster method able to accept only two inputs? What does it mean when it does? Ex: clusterX = cluster(gmfit,X); [closed]

Matlab's cluster method documentation says cluster takes in 3 arguments: T = cluster(Z,'Cutoff',C) https://www.mathworks.com/help/stats/cluster.html But line 56 inside the cluster function seems to ...
0
votes
0answers
5 views

Decompose 2D matrices into mixture of Gaussians

I have a 2D array that roughly represents a probability distribution in the 2D space. That is, all values in this 2D matrix sum up to 1. I want to decompose this 2D matrix into a sum of Gaussians. ...
4
votes
0answers
37 views

Implementation of EM algorithm confusion

Here EM algorithm manually implemented, there's a question of the implementation in R of the EM algorithm for 2 mixed gaussians. The answer has a supposedly correct implementation. However, don't the ...
1
vote
0answers
16 views

How do I properly scale the covariance matrix in a weighted Gaussian mixture model for new samples?

I am trying to implement the method for computing a Gaussian mixture model from samples with known weights as detailed in section III of: EM Algorithms for Weighted-Data Clustering with Application ...
0
votes
0answers
7 views

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture Problem description I am trying to normalize multivariate time series data. The ...
0
votes
1answer
15 views

Expected Misclustering rate

I am reading this paper on minimax clustering error rates on high-dimensional Gaussian mixtures. The authors define a metric for expected misclustering rate as follows: For a two-component ...
0
votes
0answers
33 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
0
votes
0answers
29 views

Confidence regions after fitting a 2 parameter gaussian mixture model?

Suppose I have a gaussian mixture model with 2 parameters $(u,v)$ and 2 parts. The model is $P({x_i}|u,v)=uN(x_i|\mu_1^{i} = x_i^2/v,\sigma_1^{i}) + (1-u)N(x_i|\mu_2^{i} = x_i^3/2v^2,\sigma_2^i)$. ...
4
votes
1answer
64 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
0
votes
0answers
20 views

Training Hidden Markov model with GMM, nan appears after some iterations

Problem During the training process of my continuous observation sequence data using HMM with GMM mixtures, the cost function reduces gradually and it becomes NaN ...
0
votes
0answers
38 views

Maximum likelihood convergence in mixture gaussian

Suppose there are two datasets $D_{1}$ and $D_{2}$ with same structure, which means the cluster and cluster proportion is the same. The only difference between them is that the size of $D_{1}$ is $n_{...
0
votes
0answers
15 views

How is jaccard similarity used to find the similarity between Bootstrap samples when measuring stability of EM?

Im reading the answer on "how to determine number of clusters in EM algorithm". How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results? One of the ...
0
votes
1answer
63 views

Inverse distribution of gaussian mixture

In one of the papers I've encountered, the authors propose a copula function $$ c(u_1, \ldots, u_d; \Theta) = \frac{\psi(y_1, \ldots, y_d; \Theta)}{\prod_{j=1}^{d}\psi_j (y_j)}$$ where $\psi(y_1, \...
0
votes
0answers
11 views

Biased viterbi training result

I try to use GMM-HMM model to infer the topic of sentences in a short paragraph. While instead of using normal Baum-Welch optimization, I use viterbi training as follows. I use average of word ...
0
votes
0answers
17 views

Expectation of normal RV conditional on normal mixture

Let $v\sim\text{Normal}\left(\mu,\sigma_v^2\right)$ a random variable with $\mu>0$ and $u\sim\text{Normal}\left(0,\sigma_u^2\right)$ Let $k\sim\text{Binomial}\left(N,p\right)$ a random variable ...
1
vote
1answer
45 views

Can we use a mixture of normal distributions while optimising likelihood?

Let's assume that we generate some values by a mixture of two Gaussians. Now we want to find the parameters of the two Gaussians by likelihood maximisation. One good expect that the optimisation will ...
0
votes
0answers
18 views

Interpretation of NLP pipeline for topic discovery using gaussian mixture model clustering

I built a pipeline that does the following to discover topics out of a (very big: 50k docs per ~350 terms) Term Document Matrix: Compute the TfIdf score for each Term x Document pair; Rescale each ...
0
votes
1answer
38 views

Calculation of Bayeain rule as classifier for mixture Gaussian model

Here is a paper which used a bayesian classification based on Gaussian mixture model I read many article saying that we can fit a gaussian mixture model to a data and then, based on the estimated ...
1
vote
0answers
54 views

skew G-Jensen-Shannon divergence between multivariate gaussian calculation discrepancy

I'm trying to calculate the Jensen-Shannon divergence between two multivariate Gaussians. I found a closed-form expression both for the KL divergence and JS divergence between two Gaussians in this ...
0
votes
0answers
18 views

How to forecast a time-series with a dynamic time unit?

I'm working on a forecasting problem, and I'm not sure if the data requires any transformation because the unit of time is dynamic. I'm working with an education data set where I have data on ...
0
votes
0answers
17 views

How to 'normalize' the product of two variables from Gaussian distribution?

I have two variables, x1 and x2, which are sampled from two Gaussian distributions respectively. I created an iteraction term x3 which is x1 multiply x2. Not surprisingly, x3 has very fat tail, ie., ...
2
votes
1answer
94 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
5
votes
2answers
604 views

Why do we use Gaussian distributions in Variational Autoencoder?

I still don't understand why we force the distribution of the hidden representation of a Variational Autoencoder (VAE) to follow a multivariate normal distribution. Why this specific distribution and ...
1
vote
0answers
47 views

Hidden Markov Model for classification

I have fitted a Gaussian mixture model to my data. This Gaussian mixture model is the combination of two Gaussian distributions. I call the first Gaussian distribution state 1 and the second Gaussian ...
0
votes
1answer
19 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
1
vote
1answer
867 views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
3
votes
0answers
41 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
1
vote
1answer
186 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
2
votes
1answer
184 views

Sampling from Gaussian mixture models, when are the sampled data independent?

Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions $p(x) = \sum\limits_{i = 1}^N w_i \mathcal{N}(x;\mu_i, \Sigma_i)$ where $w_i$ are the weights. Now I sample some points $\...
7
votes
2answers
905 views

Why use a Gaussian mixture model?

I am learning about Gaussian mixture models (GMM) but I am confused as to why anyone should ever use this algorithm. How is this algorithm better than other standard clustering algorithm such as $K$-...
0
votes
0answers
14 views

Imposing constraints on observation model in a HMM

I have $N$ observations ($x_1, x_2,.. ,x_N$) from a HMM with $K$ latent states. The M step for computing the observation model $\mu_k$ involves maximizing the expression: $$ L = \sum_{n=1}^{N}{ln \...
1
vote
1answer
17 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
0
votes
0answers
11 views

Can we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?

I learned from this answer that: A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A gaussian-mixture is the ...