Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0
votes
2answers
32 views

Derivation of gaussian mixture models assuming that hidden variable is known

I saw the following notes from CS229 (screenshotted below). I am confused how the two equations are equivalent. How were they able to distribute the $log$ inside the summation? I don't see how knowing ...
0
votes
0answers
25 views

How does scikit-learn handle high dimensionality in its Gaussian Mixture Model implementation?

I have a dataset of 50,000 rows that I plan to fit with scikit-learn's GMM model. The dataset has 15 features, therefore I treat each row as a vector in the space $\mathbb{R}^{15}$. My question is, ...
0
votes
0answers
15 views

Variance when sampling from a GMM? [duplicate]

I am doing some work with Gaussian mixture models and we want to find the standard deviation of samples from the model. Our current methodology is to run a Monte-Carlo sim, and just take a bunch of ...
0
votes
1answer
18 views

Upper bound on total variation between two Gaussian mixture

For two random variables $P$ and $Q$ over $R^d$ with distributions $p$ and $q$, respectively, the total variation is defined as $$ TV(P,Q)=\frac{1}{2}\int_{R^d}\ |p(x)-q(x)|dx. $$ Consider the case ...
1
vote
1answer
24 views

Question about sums of Gasussian Mixture models

This question is strongly based on the result given in HERE Due to some research I am currently conducting, I've found myself in a situation, where I deal with mixtures of gaussian densities, called ...
0
votes
0answers
18 views

Gaussian Mixture Models, application?

I'm analyzing the energy consumption behavior of a population that is increasing monthly (panel data). The population is segmented by both gender and 5 geographical locations. I gather from the data ...
3
votes
0answers
29 views

Expectation-Maximisation derivations [duplicate]

I've come across a few different sources on expectation-maximisation which I can't quite match up. The CS229 lecture 8 [1] states that the function we must write down and maximise is: $$ Q_1 = \sum_{...
1
vote
1answer
28 views

How to calculate log likelihood for gaussian mixture model

I'm trying to check the calculation of the log likelihood of a 2 component Gaussian Mixture Model using optim, but I get the wrong answer (it should return mu, sigma, alpha actual). The log ...
0
votes
0answers
11 views

Fitting a mixture model distribution to kurtotic data

I need to fit a parametric distribution to data that has non-zero (unknown) kurtosis. First I tried to fit a Pearson type VII / Student's t, but the fitting is especially poor in the two tails, ...
0
votes
0answers
7 views

Building a mixture model that fits well to the tail of a kurtotic distribution

I need to fit a distribution to data that has non-zero (unknown) kurtosis. I tried to fit a Pearson type VII / Student's t, but the fitting is especially poor in the two tails, possibly due to less ...
3
votes
1answer
59 views

Are Neural Networks Mixture Models?

To my understanding, Gaussian Mixture models are a set of parameterized gaussian distributions that collectively describe an entire, aggregate distribution. ^ from McGonagle et al Also to my ...
1
vote
2answers
40 views

Can a variational autoencoder be interpreted as a mixture of Gaussians?

In a variational autoencoder (VAE) we have an encoder network $E_{\phi}$ that maps inputs $x$ to the distribution parameters of the approximate posterior $q_{\phi}(z \vert x)$. Most commonly we model ...
0
votes
1answer
36 views

M step EM algorithm in Mixture Models. Expected value of the indicator variable under the posterior [closed]

I am not able to solve the following expectation. In the EM algorithm, the first step in the M step is to compute the expected value of $\log p(x,z)$ where $x$ are observations and $z$ indicator ...
1
vote
0answers
14 views

Mixture or Convolution

tl;dr is final paragraph at the bottom. I have read the posts explaining the differences between mixture distributions and convolutions of distributions, but am having a hard time understanding which ...
1
vote
0answers
24 views

Initialisation strategies for learning Hidden Markov Models

I used hmmlearn library to initialize an HMM (Hidden Markov Model). sampled observations from the HMM, and used the sampled data to re-estimate the parameters of ...
1
vote
0answers
13 views

Gaussian mixture models for image matrix not determining E step

I want to calculate responsibility for each of the data points, for the given MU, SIGMA and PI. ...
0
votes
0answers
10 views

Compute membership probabilities in E-step of EM algorithm with log-densities instead of densities

As an exercise I have implemented the EM algorithm for Gaussian mixtures, however, I have the problem that in high dimensions the densities of data points become so small that I get a numerical ...
0
votes
0answers
16 views

Multinomial Likelihood function with conditional probabilities drawn from Gaussian Mixtures

I have a Likelihood function that is a multinomial distribution: $p(X | \alpha, \beta) = \prod_{n=1}^N [p(x_n | \alpha)]^{I_n} [p(x_n | \beta)]^{1-I_n}$ where $I_n$ is an indicator function and both $...
0
votes
0answers
24 views

GMM: Negative BIC values decreasing with k due to small penalty

I am performing GMM clustering on 10 million datapoints with 5 features. I am trying to use the BIC score to estimate the number of clusters, however the BIC score continously decreseases as k ...
1
vote
1answer
23 views

Simulating with mixture distribution

I have fitted a gaussian mixture distribution to residuals, and now I want to simulate the residuals. However, I want the model to be independent of the time steps(have it as an input to the model). ...
1
vote
1answer
23 views

Speaker Recognition ML tasks are supervised or unsupervised?

Given the scenario: We have a speech recording from an unknown person. We have a speech recording from a known person. We have a large database of speech recordings from different persons. We would ...
0
votes
0answers
8 views

When using GMMs for speaker identification, how to add a new speaker to the already trained model?

I am using GMMs for classifying the MFCC features of voice samples for speaker identification. In the beginning I have a database of speakers which contains voice samples. I create a GaussianMixture (...
0
votes
0answers
15 views

What is typology of GMM's in Speaker Recognition and how does it differ from DNN?

I am trying to get my head around the different approaches in Speaker Recognition but I struggle to see the bigger picture. I have read the there is 2 main type of SR systems: template matching and ...
0
votes
0answers
8 views

Is this a typo in the `mclust` exposition?

I'm reading about the mclust package for gaussian mixture models. I want to understand its statistical assumptions. https://link.springer.com/content/pdf/10.1007/...
4
votes
1answer
84 views

Derivation of maximum likelihood for a Gaussian mixture model

I'm working my way through the derivation of EM in Bishop (p. 435). I'm stuck trying to derive to MLE for $\mu_k$ for the gaussian mixture model. Basically I get an extra sum in the numerator. For ...
0
votes
0answers
5 views

Regularise space between points with differentiability

My problem is in one dimension. I have points $(x_i)$ that are mixture of gaussians (one or more). I want to regularise the space between the center of each gaussians in the following way : If $c_j$ ...
0
votes
1answer
25 views

What is the Q distribution in expectation maximization in the following explanation?

I am reading a blog on expectation maximization - http://krasserm.github.io/2019/11/21/latent-variable-models-part-1/ Here, I encounter the following expression: When you look at the above ...
1
vote
1answer
55 views

Modeling time series with Gaussian Mixture Model

I'm reading Song and Wang's paper on incremental estimation of GMM for online data streaming clustering. I assumed that we could apply the same idea to model time series, as a time series is a data ...
1
vote
0answers
45 views

Numerical Integration with respect to a mixture of Normals [closed]

I have a likelihood function that contains an integral of a latent parameter. I would like to numerically integrate it using Monte Carlo, as in, $L = \prod_{i=1}^N \int f(X, \tilde{\theta}; \beta) d ...
0
votes
0answers
43 views

Goodness of fit measure for GMM similar to the likelihood function

Suppose we have two trained models for a specific color in an image: 1-Multivariate Gaussian Model (MVG), 2-Gaussian Mixture (GMM). I need to identify the most probable region which then be used for ...
2
votes
2answers
89 views

interpretation of the estimated parameters of a gaussian mixture model

I need to find/fit a model for the color of an object. Suppose its color is generally yellow and we have 10000-by-3 data which are pixel values for R, G, B channels. Firstly I choose a Multivariate ...
0
votes
0answers
26 views

Ideas for Clustering Feedback Survey (GMM?)

I have a huge amount of written feedback from an employee survey (about 10.000 answers), which I would like to cluster (such as dissatisfaction with the boss / the health system / the work-life-...
1
vote
2answers
42 views

fitting a Gaussian mixture with a constraint in python

Suppose I have data and I want to fit a two component Gaussian mixture to it. I don't know how to do it in python but worse than that is that I have an additional constraint that the mean of one ...
1
vote
1answer
29 views

Gaussian Mixture model - Penalized log-likelihood in EM algorithm not monotone increasing

I am working on a multivariate Gaussian Mixture Model in R. The goal is to do regularized clustering on the data, where each component represents a cluster. I wrote an EM algorithm to maximize a ...
0
votes
1answer
43 views

Mixture of normal distributions, all with the same variance and with normally distributed means

Say you have a mixture distribution with the following properties: It is made up of multiple normal distributions The variances of all those component normal distributions are all the same The means ...
0
votes
0answers
15 views

How to show that GMM has the same assignment done as k-mean when the covariance is 0?

Given a Gaussian mixture \begin{equation}p(x) = \sum_{k=1}^{K}\pi_kN(x:M_k,\sum)\end{equation} with fixed uniform mixing weights $\pi_k = 1/k$ and has the same fixed isotropic covariance matrix $\sum ...
4
votes
1answer
108 views

the approximation power of Gaussian mixture models?

What are the probability density functions that GMM can approximate? a reference in appreciated about this.
0
votes
0answers
43 views

Extending HMM with Gaussian Emissions to GMM

In the notation/language of HMMs, say $h_{1:T_i}^i$ be the hidden states, and $v_{1:T_i}^i$ be the observations where $i=1,\ldots,n$ denote each training set. Let each mutlivariate observation $v_t \...
1
vote
1answer
184 views

Compute mean and variance of mixture of Gaussians given mean/variance of component gaussians [duplicate]

Given $N$ means and variances $\{\mu_1,\mu_2,....\mu_N\}$ , $\{\sigma_1^2,\sigma_2^2,....\sigma_N^2 \}$ ,and the fact that combined they make a gaussian mixture, how do I compute for that mixture $M$, ...
1
vote
1answer
15 views

Fitting mixture model on data with duplicate values

What is the correct procedure to fit finite mixture models on data with many duplicate values using EM? Let's say I have N(0,1) distributed data and try to fit a 2 component mixture using EM. There ...
1
vote
0answers
45 views

testing whether data comes from a bi-modal distribution (python) [duplicate]

I have a variable which seems to be a mix of two Gaussian distributions (it is bi-modal with each mode looking normally distributed). I would like to identify anomalous samples. So my idea is to ...
2
votes
0answers
66 views

Can someone verify if the following Bayesian Information Criterion (BIC) model selection algorithm is correct for Gaussian mixture models?

I am trying to find an automated way of picking the number of clusters $K \in \mathbb{N}$ for unsupervised learning scenarios, specifically for GMM. I was suggested to use something called the "...
2
votes
0answers
70 views

Graphical model of the Gaussian mixture: where is n?

TL;DR: Where are the occupation numbers in the Graphical model of the GMM? I am implementing a Finite (to be adapted to infinite later) Gaussian Mixture Model. I am using the Gibbs sampler-ready ...
0
votes
1answer
27 views

Ponderate two gaussian mixture

I have 2 independent random variables $X$ and $Y$ with gaussian mixture distribution like: $$f(x) = \sum_{i=1}^{m} \phi_{X,i} \mathcal{N}(\mu_{X,i} , \sigma_{X,i}^{2})$$ $$f(y) = \sum_{i=1}^{m} \phi_{...
2
votes
0answers
125 views

Efficient sampling from a multivariate Gaussian Mixture distribution for a given CDF level

I have a multivariate Gaussian Mixture (GM) distribution. I am wondering if there is any more efficient way of drawing samples (i.e., identify the iso-surface) from a multivariate Gaussian Mixture ...
0
votes
0answers
24 views

expectation of normal mixture density by given cut off point

I am self-study multivariate statistics with book "A First Course in Multivariate Statistics", I don't know how to solve the problem from section 2.8 of exercise 11 which states that: consider a ...
2
votes
2answers
53 views

EM-algorithm for two clusters (when one of the distributions is uniform)

I am having a hard time with the EM-algorithm. Here's the problem that I am trying to solve. Dealing with noisy annotations is a common problem in computer vision, especially when using ...
0
votes
0answers
35 views

GDA producing negative covariance determinant

I'm running Gaussian Discriminant Analysis across a large set of examples (~80k) in $\mathbb{R}^{8}$. I know that the covariance matrix $\Sigma$ is, by definition, positive semi-definite, which means ...
0
votes
0answers
160 views

Comparing 2 mixture models using mixtools

I have 2 mixture models I'd like to compare. Specifically, I want to compare lamda (i.e. proportion/area under each distribution) as it looks like there are differences there. Is this possible? ...
0
votes
0answers
24 views

Analysing faithful dataset in R using GMM

I have got a project on analysing the faithful data in R found in the package "datasets" and called using data(faithful) which is the data set off eruption time and waiting time of the Old Faithful ...

1
2 3 4 5
10