Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
3
votes
1answer
16 views

the approximation power of Gaussian mixture models?

What are the probability density functions that GMM can approximate? a reference in appreciated about this.
0
votes
0answers
9 views

Extending HMM with Gaussian Emissions to GMM

In the notation/language of HMMs, say $h_{1:T_i}^i$ be the hidden states, and $v_{1:T_i}^i$ be the observations where $i=1,\ldots,n$ denote each training set. Let each mutlivariate observation $v_t \...
1
vote
1answer
22 views

Compute mean and variance of mixture of Gaussians given mean/variance of component gaussians [duplicate]

Given $N$ means and variances $\{\mu_1,\mu_2,....\mu_N\}$ , $\{\sigma_1^2,\sigma_2^2,....\sigma_N^2 \}$ ,and the fact that combined they make a gaussian mixture, how do I compute for that mixture $M$, ...
1
vote
1answer
11 views

Fitting mixture model on data with duplicate values

What is the correct procedure to fit finite mixture models on data with many duplicate values using EM? Let's say I have N(0,1) distributed data and try to fit a 2 component mixture using EM. There ...
1
vote
0answers
38 views

testing whether data comes from a bi-modal distribution (python) [duplicate]

I have a variable which seems to be a mix of two Gaussian distributions (it is bi-modal with each mode looking normally distributed). I would like to identify anomalous samples. So my idea is to ...
2
votes
0answers
16 views

Can someone verify if the following Bayesian Information Criterion (BIC) model selection algorithm is correct for Gaussian mixture models?

I am trying to find an automated way of picking the number of clusters $K \in \mathbb{N}$ for unsupervised learning scenarios, specifically for GMM. I was suggested to use something called the "...
2
votes
0answers
50 views

Graphical model of the Gaussian mixture: where is n?

TL;DR: Where are the occupation numbers in the Graphical model of the GMM? I am implementing a Finite (to be adapted to infinite later) Gaussian Mixture Model. I am using the Gibbs sampler-ready ...
0
votes
1answer
26 views

Ponderate two gaussian mixture

I have 2 independent random variables $X$ and $Y$ with gaussian mixture distribution like: $$f(x) = \sum_{i=1}^{m} \phi_{X,i} \mathcal{N}(\mu_{X,i} , \sigma_{X,i}^{2})$$ $$f(y) = \sum_{i=1}^{m} \phi_{...
1
vote
0answers
118 views

Efficient sampling from a multivariate Gaussian Mixture distribution for a given CDF level

I have a multivariate Gaussian Mixture (GM) distribution. I am wondering if there is any more efficient way of drawing samples (i.e., identify the iso-surface) from a multivariate Gaussian Mixture ...
0
votes
0answers
22 views

expectation of normal mixture density by given cut off point

I am self-study multivariate statistics with book "A First Course in Multivariate Statistics", I don't know how to solve the problem from section 2.8 of exercise 11 which states that: consider a ...
2
votes
2answers
47 views

EM-algorithm for two clusters (when one of the distributions is uniform)

I am having a hard time with the EM-algorithm. Here's the problem that I am trying to solve. Dealing with noisy annotations is a common problem in computer vision, especially when using ...
0
votes
0answers
20 views

GDA producing negative covariance determinant

I'm running Gaussian Discriminant Analysis across a large set of examples (~80k) in $\mathbb{R}^{8}$. I know that the covariance matrix $\Sigma$ is, by definition, positive semi-definite, which means ...
0
votes
0answers
16 views

Comparing 2 mixture models using mixtools

I have 2 mixture models I'd like to compare. Specifically, I want to compare lamda (i.e. proportion/area under each distribution) as it looks like there are differences there. Is this possible? ...
0
votes
0answers
10 views

Analysing faithful dataset in R using GMM

I have got a project on analysing the faithful data in R found in the package "datasets" and called using data(faithful) which is the data set off eruption time and waiting time of the Old Faithful ...
0
votes
0answers
15 views

Marginal distributions of a mixture of two Gaussian distributions

In the Mathematics for Machine Learning book (Section 6.4), the author introduces a mixture of two Gaussian distributions as $$0.4 \mathcal{N}\Bigg(\begin{bmatrix}10 \\ 2\end{bmatrix}, \begin{...
0
votes
1answer
33 views

Feature importance for Gaussian Mixture Model

After GMM clustering I've got quite logical clusterization and the question is how can measure the importance of the exact feature in clustering.
0
votes
0answers
46 views

Difference between GMM and HMM

From what I understand: GMM is a probabilistic model which can model N sub population normally distributed. Each component in GMM is a Gaussian distribution. HMM is a statistical Markov model with ...
0
votes
0answers
16 views

Approximate inverse of a Gaussian Process

I'm using a GP in order to learn the transition function of a continuous Markov Decision Process, i.e. P(s'|s,a). This works reasonably well, but I'm now also ...
0
votes
1answer
15 views

what does mixture mean in the context of Gaussian Naive Bayes classifier?

This CMU Machine Learning Text Book is talking about naive bayes. Of course we must also estimate the priors on Y as well $π_k = P(Y = y_k)$ The above model summarizes a Gaussian Naive Bayes ...
0
votes
0answers
17 views

sampling question in Gibbs sampling for a Gaussian mixture model

I have some confusions regarding the Gibbs sampling step for the following mixture model: consider a mixture model of the following generative process: $\theta \sim Dir(\alpha) $ (global hidden ...
1
vote
0answers
13 views

Reference request: statement about jointly Gaussian RV

I found a theorem online https://people.eecs.berkeley.edu/~ananth/223Spr07/jointlygaussian.pdf Is there a secondary reference for this theorem?
1
vote
1answer
56 views

Using AIC/BIC within cross-validation for likelihood based loss functions

For a course I am teaching, I am having my students fit a Gaussian mixture model using MLEs via the EM algorithm to a bivariate dataset. I have asked the students to use use cross-validation to choose ...
0
votes
0answers
10 views

Error in flexmix gaussian mixture model (all components removed)

I am running gaussian mixture models with flexmix R package. The function works fine with up to k=20, but when I try k>20 it gives the following error: error in flxfit(model = model, concomitant = ...
0
votes
1answer
26 views

How many parameters are present in a (general) discrete mixture of five normal distributions?

What is the minimal amount of parameters that can be retained in a particular context?
0
votes
0answers
35 views

Determining Link for GLM

I am having a great deal of difficulty understanding how to use the Generalized linear model for my data set. The response variable of interest is hatch success of sea turtles, which is a %. The ...
1
vote
0answers
14 views

How to fit mixture of gaussians with identical mean?

Say I have data generated by a mixture of gaussians whose components have the same mean, but very different covariances, like the one generated by this code: ...
1
vote
1answer
34 views

What does the y parameter in .fit() of scikit-learn's Gaussian Mixture Model do?

From my understanding, Gaussian Mixture Models are an unsupervised method and can perform clustering similar to k-means. In the scitkit-learn implementation of GMM, the .fit() function (and ....
1
vote
1answer
143 views

Conditional Probability - Mixture Model

I know that the likelihood in a p-dimensional Gaussian mixture model is given by $$ p(s|\theta) = \sum_{b_1 = 0}^1\cdots\sum_{b_p = 0}^1\left[ \prod_{i=1}^pw^{1-b_i}(1-w)^{1-b_i}\right]\phi_p(s|\mu(b,...
0
votes
0answers
10 views

Probabilistic Models, what do they infer?

As per my understanding, Mixture Models such as GMM, Probabilistic Models such as Variation Autoencoder, they explain the latent space behind the features. But how they turn from learning latent space ...
0
votes
1answer
25 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
2
votes
1answer
108 views

Distribution of Distributions | percent point function

I have the following normal distribution (Primary Distribution): Mean = 7 Sigma = 0.5 and a list of other normal distributions (Secondary Distributions): (python list of tupels, each tupel ...
2
votes
1answer
88 views

Figueiredo and Jain's Gaussian mixture EM convergence criterion

I have implemented and been playing around Figueiredo & Jain 's trainer in this paper http://www.lx.it.pt/~mtf/IEEE_TPAMI_2002.pdf for Gaussian mixture. Fig. 2 in the paper depicts the full ...
0
votes
0answers
57 views

Gibbs Sampler for GMM

In Rasmussen's paper it is introduced a Gibbs sampler to make inference about a standard Gaussian Mixture Model. To simplify, assume the 1-d case with basic hierarchical structure, that is: $x_i|...
0
votes
2answers
46 views

Convergence of EM algorithm

I am aware that EM eventually converges. However, I still have some confusions regarding this property: 1: As far as I am aware, HMM, Gaussian mixture model and MCMC can converge and all of them use ...
0
votes
0answers
87 views

what is wrong with my Gaussian Mixture density estimation fitting (Python)?

I have a data set (1D) link: dataset, which has values ranging from 21,000 to 8,000,000. When i plot histogram of the log values, i can see there are two peaks, roughly. I tried to fit Gaussian ...
1
vote
0answers
56 views

Closed form for Finite Gaussian Mixture Model when weights are known and prior variance can be 0

Suppose I have a normal likelihood $x|\theta \sim N(\theta, \sigma^2_{known})$ where the variance is known and a mixture prior $\theta \sim p * N(\mu_1, \sigma^2_1) + (1-p) * N(\mu_2, \sigma^2_2)$, ...
0
votes
2answers
136 views

What is the interpretation of the weights in the GMM?

GMM is $p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the interpretation of the weights here? Do ...
1
vote
0answers
26 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
0
votes
0answers
29 views

How to deduct the complete likelihood of mixture normal in EM algorithm

We have the well known complete likelihood of mixture normal in EM algorithm: Here $Z$ is a random variable that it has probability $\pi_k$ to choose k-th normal variable $X_k:N(\mu_k,\sigma_k).$ We ...
0
votes
2answers
109 views

Suggest a model for this dataset

I have a time series data set (the Old Faithful geyser data available here: http://www.gatsby.ucl.ac.uk/teaching/courses/ml1-2012/geyser.txt). Plotting the eruption duration on the x axis and the ...
2
votes
2answers
62 views

Interpreting mixture of Gaussians (Variational Inference)

I've recently stated reading about mixture models and variational inference in this excellent paper, but I'm having troubles dissecting the models described, and have a couple of questions. Please see ...
2
votes
2answers
44 views

Can Gaussian Mixture Model Clustering tell me something about the distribution of my data?

I have 10,000 vectors originating from 5 separate classes (2,000 each). I use Gaussian Mixture Model clustering (in Python) to cluster the 10,000 vectors, telling the algorithm to cluster the data ...
1
vote
2answers
22 views

Why is Matlab's cluster method able to accept only two inputs? What does it mean when it does? Ex: clusterX = cluster(gmfit,X); [closed]

Matlab's cluster method documentation says cluster takes in 3 arguments: T = cluster(Z,'Cutoff',C) https://www.mathworks.com/help/stats/cluster.html But line 56 inside the cluster function seems to ...
1
vote
0answers
12 views

Decompose 2D matrices into mixture of Gaussians

I have a 2D array that roughly represents a probability distribution in the 2D space. That is, all values in this 2D matrix sum up to 1. I want to decompose this 2D matrix into a sum of Gaussians. ...
4
votes
0answers
58 views

Implementation of EM algorithm confusion

Here EM algorithm manually implemented, there's a question of the implementation in R of the EM algorithm for 2 mixed gaussians. The answer has a supposedly correct implementation. However, don't the ...
1
vote
0answers
41 views

How do I properly scale the covariance matrix in a weighted Gaussian mixture model for new samples?

I am trying to implement the method for computing a Gaussian mixture model from samples with known weights as detailed in section III of: EM Algorithms for Weighted-Data Clustering with Application ...
0
votes
0answers
10 views

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture Problem description I am trying to normalize multivariate time series data. The ...
0
votes
1answer
22 views

Expected Misclustering rate

I am reading this paper on minimax clustering error rates on high-dimensional Gaussian mixtures. The authors define a metric for expected misclustering rate as follows: For a two-component ...
0
votes
0answers
47 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
0
votes
0answers
37 views

Confidence regions after fitting a 2 parameter gaussian mixture model?

Suppose I have a gaussian mixture model with 2 parameters $(u,v)$ and 2 parts. The model is $P({x_i}|u,v)=uN(x_i|\mu_1^{i} = x_i^2/v,\sigma_1^{i}) + (1-u)N(x_i|\mu_2^{i} = x_i^3/2v^2,\sigma_2^i)$. ...