Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
1
vote
1answer
17 views

How to get number of iterations in EM-algorithm using R mclust gaussian mixture model

I am clustering data using the mclust function from the R mclust package. I am struggling to get the number of iterations the EM ...
1
vote
1answer
42 views

Bayes' theorem in Gaussian mixture model for $p(z_i = k | x_i, \mu_k,\Sigma_k)$

Given the $k$th Gaussian distribution $N \sim (\mu_k, \Sigma_k)$, the probability that $x_i$ generated from this Gaussian $k$ can be found via Bayes' rule $$\begin{align}p(z_i = k | x_i,\mu_k, \...
0
votes
1answer
25 views

Gaussian Mixture Model $p(x_i | z_i = k)$ a likelihood or probability?

In Gaussian Mixture models, the probability of observing the data $x$ given that it was generated from $M$ gaussian models is given by the following equation $$p(x) = \sum_{k=1}^m p(x|z=k)p(z = k)$$ ...
0
votes
1answer
23 views

Probability Density Function and Maximum Likelihood Estimation for Multinomial Logistic Regression and GMM

I have some confusion about a few very basic concepts and terminology. Let's assume we have two models for classification, a multinomial logistic regression (MLR) model and a GMM classifier. I'm not ...
0
votes
0answers
25 views

Derivation of maximum likelihood estimation of Gaussian mixture model

I want to derive the following formula. The meaning of each expression is as follows It's easy to solve with Kronecker's delta.
0
votes
0answers
25 views

Confidence intervals for mixture of Gaussian distributions

I have a mixture distribution of 2 Gaussians. Here, the left has a weight of 0.1 and the right has a weight of 0.9. In this example, they have identical $\sigma$, but that may not always be the case....
0
votes
1answer
21 views

Order and interpretation of Gaussian Mixture Model with strong overlap between components

Most examples for Gaussian Mixture Models (GMMs) employ datasets with fairly obvious underlying structure (well-separated clusters). How should one determine the order of a GMM (and interpret the ...
0
votes
0answers
11 views

How to obtain equation for evidence given equation for joint density of latent and observed variables (Variational Inference)

I am working through the following review paper: https://arxiv.org/pdf/1601.00670.pdf and would like to gain some insight into arriving at the formulae for joint density of latent and observed ...
0
votes
0answers
8 views

Mixture model for decomposing bimodal or multimodal distributions

The gaussian mixture model (GMM) is fed mixture components or features whose time series each have differing means and variances from one another, but are unimodal (have one mode) with each component ...
2
votes
1answer
55 views

Meaning of Gaussian mixture weights?

other posts with similar title do not actually ask what's in their title, so I ask here: What is the meaning of the weights in the Gaussian Mixture Model (GMM)? Does the GMM weight more heavily to ...
2
votes
1answer
41 views

Can a Gaussian Mixture model be fit with a continuous response variable?

Does the Gaussian Mixture model require binary and multiclass response/target variable (classification), or can the target vector consist of all real numbers (continuous variable, regression)? Why is ...
0
votes
0answers
9 views

Can Gaussian mixture models help an algorithm target a specific cluster?

In the chart below is a Gaussian Mixture model (GMM) based on three time series or datasets that the model was able to easily cluster into three different colored classes. Class 1 is the blue ellipse ...
0
votes
2answers
70 views

Probability or likelihood under normal distribution(s)?

I've modeled my data with a mixture model of two gaussians centered at approximately 0.33 and 0.5, respectively. Now I want to "assign" a probability to each data point that it belongs to ...
0
votes
0answers
20 views

Noise learning in GMM

The Gaussian mixture model is a parametric model that learns the underlying distribution of data. If data contains the noisy attributes and noisy samples then will the model learn that noisy data too ...
1
vote
1answer
56 views

Calculate log likelihood of mixture of gaussians “by hand” in R

I'm trying to ensure the the method of calculating log likelihood for a model produced using mixtools vs a model produced using MLE estimates of mu and sigma are the same. The best way I can think of ...
3
votes
1answer
167 views

Derivation of M step for Gaussian mixture model

Summary So to summarize my question, how can I take \begin{align} = \sum_{i=1}^{n}W_{i1} \left(log (1-\sum_{j=2}^{K}\pi_j) -\frac{1}{2} log(|\Sigma_1|) -\frac{d}{2} log(2\pi) -\frac{1}{2}(x_i-\mu_1)^{...
0
votes
0answers
16 views

Gaussian mixture model parameter updates derivation?

I've been following a helpful tutorial and I'm trying to understand the parameter updates. For example, the mu_k parameter update is below. I'm unsure why the sum(Bk) does not cancel out as it's in ...
0
votes
0answers
13 views

Gaussian Mixture Model Clustering - cluster means are assigned to a different cluster

I ran a gaussian mixture model with 7 clusters on my data. My data has been PCA transformed with 200 components. Then I extracted the means of each cluster and applied the predict_proba function on ...
0
votes
0answers
11 views

Are there any advantages of k-means over Gaussian Mixture (Expectation minimization)?

In other words, if I already included Gaussian mixture into the analysis, does it make sense to add also k-means, as GM clustering is a generalization of k-means?
0
votes
0answers
4 views

Id+s this kind of data (picture) suitable for gaussian mixture models)?

I have a dataset with two features A and B. I was wondering if this kind of data is suitable for GMM since distributions don't follow exact gaussian. As you can each feature has two or more "...
2
votes
1answer
73 views

Difference Between Latent Class Analysis and Mixture Models

I have been trying to look into latent class analysis and don't exactly understand what it is. Is it basically the expectation maximization using and analyzing the classes formed? The resources on the ...
1
vote
1answer
39 views

Understanding Loss functions in Stacked Capsule Autoencoders

I was reading Stacked Capsule Autoencoder paper published by Geoff Hinton's group last year in NIPS. While reading section 2.1 about constellation autoencoders I couldn't understand how the expression ...
0
votes
2answers
36 views

Derivation of gaussian mixture models assuming that hidden variable is known

I saw the following notes from CS229 (screenshotted below). I am confused how the two equations are equivalent. How were they able to distribute the $log$ inside the summation? I don't see how knowing ...
0
votes
0answers
41 views

How does scikit-learn handle high dimensionality in its Gaussian Mixture Model implementation?

I have a dataset of 50,000 rows that I plan to fit with scikit-learn's GMM model. The dataset has 15 features, therefore I treat each row as a vector in the space $\mathbb{R}^{15}$. My question is, ...
0
votes
0answers
16 views

Variance when sampling from a GMM? [duplicate]

I am doing some work with Gaussian mixture models and we want to find the standard deviation of samples from the model. Our current methodology is to run a Monte-Carlo sim, and just take a bunch of ...
0
votes
1answer
34 views

Upper bound on total variation between two Gaussian mixture

For two random variables $P$ and $Q$ over $R^d$ with distributions $p$ and $q$, respectively, the total variation is defined as $$ TV(P,Q)=\frac{1}{2}\int_{R^d}\ |p(x)-q(x)|dx. $$ Consider the case ...
1
vote
1answer
33 views

Question about sums of Gasussian Mixture models

This question is strongly based on the result given in HERE Due to some research I am currently conducting, I've found myself in a situation, where I deal with mixtures of gaussian densities, called ...
0
votes
0answers
20 views

Gaussian Mixture Models, application?

I'm analyzing the energy consumption behavior of a population that is increasing monthly (panel data). The population is segmented by both gender and 5 geographical locations. I gather from the data ...
3
votes
0answers
30 views

Expectation-Maximisation derivations [duplicate]

I've come across a few different sources on expectation-maximisation which I can't quite match up. The CS229 lecture 8 [1] states that the function we must write down and maximise is: $$ Q_1 = \sum_{...
1
vote
1answer
31 views

How to calculate log likelihood for gaussian mixture model

I'm trying to check the calculation of the log likelihood of a 2 component Gaussian Mixture Model using optim, but I get the wrong answer (it should return mu, sigma, alpha actual). The log ...
0
votes
0answers
12 views

Fitting a mixture model distribution to kurtotic data

I need to fit a parametric distribution to data that has non-zero (unknown) kurtosis. First I tried to fit a Pearson type VII / Student's t, but the fitting is especially poor in the two tails, ...
0
votes
0answers
11 views

Building a mixture model that fits well to the tail of a kurtotic distribution

I need to fit a distribution to data that has non-zero (unknown) kurtosis. I tried to fit a Pearson type VII / Student's t, but the fitting is especially poor in the two tails, possibly due to less ...
4
votes
1answer
74 views

Are Neural Networks Mixture Models?

To my understanding, Gaussian Mixture models are a set of parameterized gaussian distributions that collectively describe an entire, aggregate distribution. ^ from McGonagle et al Also to my ...
1
vote
2answers
70 views

Can a variational autoencoder be interpreted as a mixture of Gaussians?

In a variational autoencoder (VAE) we have an encoder network $E_{\phi}$ that maps inputs $x$ to the distribution parameters of the approximate posterior $q_{\phi}(z \vert x)$. Most commonly we model ...
0
votes
1answer
44 views

M step EM algorithm in Mixture Models. Expected value of the indicator variable under the posterior [closed]

I am not able to solve the following expectation. In the EM algorithm, the first step in the M step is to compute the expected value of $\log p(x,z)$ where $x$ are observations and $z$ indicator ...
1
vote
0answers
24 views

Mixture or Convolution

tl;dr is final paragraph at the bottom. I have read the posts explaining the differences between mixture distributions and convolutions of distributions, but am having a hard time understanding which ...
1
vote
0answers
28 views

Initialisation strategies for learning Hidden Markov Models

I used hmmlearn library to initialize an HMM (Hidden Markov Model). sampled observations from the HMM, and used the sampled data to re-estimate the parameters of ...
1
vote
0answers
13 views

Gaussian mixture models for image matrix not determining E step

I want to calculate responsibility for each of the data points, for the given MU, SIGMA and PI. ...
0
votes
0answers
12 views

Compute membership probabilities in E-step of EM algorithm with log-densities instead of densities

As an exercise I have implemented the EM algorithm for Gaussian mixtures, however, I have the problem that in high dimensions the densities of data points become so small that I get a numerical ...
0
votes
0answers
21 views

Multinomial Likelihood function with conditional probabilities drawn from Gaussian Mixtures

I have a Likelihood function that is a multinomial distribution: $p(X | \alpha, \beta) = \prod_{n=1}^N [p(x_n | \alpha)]^{I_n} [p(x_n | \beta)]^{1-I_n}$ where $I_n$ is an indicator function and both $...
0
votes
0answers
53 views

GMM: Negative BIC values decreasing with k due to small penalty

I am performing GMM clustering on 10 million datapoints with 5 features. I am trying to use the BIC score to estimate the number of clusters, however the BIC score continously decreseases as k ...
1
vote
1answer
34 views

Simulating with mixture distribution

I have fitted a gaussian mixture distribution to residuals, and now I want to simulate the residuals. However, I want the model to be independent of the time steps(have it as an input to the model). ...
1
vote
1answer
27 views

Speaker Recognition ML tasks are supervised or unsupervised?

Given the scenario: We have a speech recording from an unknown person. We have a speech recording from a known person. We have a large database of speech recordings from different persons. We would ...
0
votes
0answers
11 views

When using GMMs for speaker identification, how to add a new speaker to the already trained model?

I am using GMMs for classifying the MFCC features of voice samples for speaker identification. In the beginning I have a database of speakers which contains voice samples. I create a GaussianMixture (...
0
votes
0answers
15 views

What is typology of GMM's in Speaker Recognition and how does it differ from DNN?

I am trying to get my head around the different approaches in Speaker Recognition but I struggle to see the bigger picture. I have read the there is 2 main type of SR systems: template matching and ...
0
votes
0answers
10 views

Is this a typo in the `mclust` exposition?

I'm reading about the mclust package for gaussian mixture models. I want to understand its statistical assumptions. https://link.springer.com/content/pdf/10.1007/...
4
votes
1answer
124 views

Derivation of maximum likelihood for a Gaussian mixture model

I'm working my way through the derivation of EM in Bishop (p. 435). I'm stuck trying to derive to MLE for $\mu_k$ for the gaussian mixture model. Basically I get an extra sum in the numerator. For ...
0
votes
0answers
5 views

Regularise space between points with differentiability

My problem is in one dimension. I have points $(x_i)$ that are mixture of gaussians (one or more). I want to regularise the space between the center of each gaussians in the following way : If $c_j$ ...
0
votes
1answer
27 views

What is the Q distribution in expectation maximization in the following explanation?

I am reading a blog on expectation maximization - http://krasserm.github.io/2019/11/21/latent-variable-models-part-1/ Here, I encounter the following expression: When you look at the above ...
1
vote
1answer
106 views

Modeling time series with Gaussian Mixture Model

I'm reading Song and Wang's paper on incremental estimation of GMM for online data streaming clustering. I assumed that we could apply the same idea to model time series, as a time series is a data ...

1
2 3 4 5
11