Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
2
votes
1answer
4k views

Fit mixture of distributions to your time-series data in R

I have time-series data containing 1440 observations and the plot of the data is I want to fit the Gaussian Mixture Models (GMM) to the above plot, and for the same I am using Mclust function of ...
19
votes
1answer
13k views

Different covariance types for Gaussian Mixture Models

While trying Gaussian Mixture Models here, I found these 4 types of covariances. ...
20
votes
2answers
33k views

EM algorithm manually implemented

I want to implement the EM algorithm manually and then compare it to the results of the normalmixEM of mixtools package. Of ...
25
votes
2answers
21k views

If k-means clustering is a form of Gaussian mixture modeling, can it be used when the data are not normal?

I'm reading Bishop on EM algorithm for GMM and the relationship between GMM and k-means. In this book it says that k-means is a hard assign version of GMM. I'm wondering does that imply that if the ...
12
votes
1answer
7k views

Mclust model selection

The R package mclust uses BIC as a criteria for cluster model selection. From my understanding, a model with the lowest BIC should be selected over other models (if ...
3
votes
2answers
7k views

Significant difference between time series - Can I do this?

I'd like to know whether the solution proposed below is valid/acceptable and any justification available. We have two biological conditions, and for each condition we measured 3 time series, so at ...
18
votes
2answers
4k views

Why is optimizing a mixture of Gaussian directly computationally hard?

Consider the log likelihood of a mixture of Gaussians: $$l(S_n; \theta) = \sum^n_{t=1}\log f(x^{(t)}|\theta) = \sum^n_{t=1}\log\left\{\sum^k_{i=1}p_i f(x^{(t)}|\mu^{(i)}, \sigma^2_i)\right\}$$ I was ...
10
votes
3answers
6k views

2-Gaussian mixture model inference with MCMC and PyMC

The problem I want fit the model parameters of a simple 2-Gaussian mixture population. Given all the hype around Bayesian methods I want to understand if for this problem Bayesian inference is a ...
8
votes
2answers
4k views

What is “mixture” in a gaussian mixture model

We often study Gaussian Mixture model as a useful model in machine learning and its applications. What is the physical significance of this "Mixture"? Is it used because a Gaussian Mixture Model ...
2
votes
2answers
864 views

Compute quantile function from a mixture of Normal distribution

I have this mixture of normal distribution: $$X \sim \frac{1}{2}\mathcal{N}(\mu_{x_1}=10,\,\sigma_{x_1}^{2}=1)+\frac{1}{2}\mathcal{N}(\mu_{x_2}=13,\,\sigma_{x_2}^{2}=1)$$ How can i compute the ...
10
votes
1answer
1k views

Simulate from a truncated mixture normal distribution

I want to simulate a sample from a mixture normal distribution such that $$p\times\mathcal{N}(\mu_1,\sigma_1^2) + (1-p)\times\mathcal{N}(\mu_2,\sigma_2^2) $$ is restricted to the interval $[0,1]$ ...
1
vote
1answer
634 views

My MCMC do not overlap : Mixturemodel with JAGS and R

I fitted a JAGS model and I have those results : My questions are: Why do my chains not overlap, and how can I fix that? I used the following method: My model is a mixture Gaussian model of two ...
35
votes
5answers
45k views

Clustering a dataset with both discrete and continuous variables

I have a dataset X which has 10 dimensions, 4 of which are discrete values. In fact, those 4 discrete variables are ordinal, i.e. a higher value implies a higher/better semantic. 2 of these discrete ...
15
votes
5answers
8k views

Singularity issues in Gaussian mixture model

In chapter 9 of the book Pattern recognition and machine learning, there is this part about Gaussian mixture model: To be honest I don't really understand why this would create a singularity. Can ...
5
votes
1answer
3k views

Ftting a mixture of two Gaussians

I want to fit a mixture of two gaussian densities to my financial data. The data can be found here: http://uploadeasy.net/upload/2a7mw.rar the variable is called dat. The probability density of a ...
13
votes
1answer
7k views

Quantiles from the combination of normal distributions

I have information on the distributions of anthropometric dimensions (like shoulder span) for children of different ages. For each age and dimension, I have mean, standard deviation. (I also have ...
8
votes
1answer
3k views

K-means as a limit case of EM algorithm for Gaussian mixtures with covariances $\epsilon^2 I$ going to $0$

My goal is to see that K-means algorithm is in fact Expectation-Maximization algorithm for Gaussian mixtures in which all components have covariance $\sigma^2 I$ in the limit as $\lim_{\sigma \to 0}$. ...
7
votes
4answers
14k views

How to train a Gaussian mixture hidden Markov model?

I want to build a hidden Markov model (HMM) with continuous observations modeled as Gaussian mixtures (Gaussian mixture model = GMM). The way I understand the training process is that it should be ...
7
votes
1answer
1k views

Difference between GMM classification and QDA

I know that every class has the same covariance matrix $\Sigma$ in linear discriminant analysis (LDA), and in quadratic discriminant analysis (QDA) they are different. When using gaussian mixture ...
1
vote
1answer
16k views

Generate sample data from Gaussian mixture model [duplicate]

I am given the values for mean, co-variance, initial_weights for a mixture of Gaussian Models. Now how can I generate samples given those: In brief, I need a function like ...
5
votes
2answers
3k views

Sum of Gaussian mixture and Gaussian scale mixture

What will be the distribution of the sum of two independent random variables, say $X$ and $Y$, when $X$ has a Gaussian mixture distribution (when we take Gaussian distribution with different location ...
3
votes
1answer
851 views

Is there a numerical solution to a mixture model of two normal distributions?

I'm building a mixture model with the two normal distributions $\mathcal{N}(\mu_1,\sigma_{1}^{2})$ and $\mathcal{N}(\mu_2,\sigma_{2}^{2})$. So, the density function is $$ f(x) = p_1 N(x; \mu_1, \...
3
votes
1answer
191 views

What is the link (discrepancy?) between these PDF/CDF and p-value distributions?

I have created a mixed distribution model comprising 80% $H_0$ plus 20% $H_1$ to illustrate the link between the expected proportions of true and false positives and negatives in the PDF, CDF and p-...
1
vote
2answers
349 views

Finding out if your data belongs to normal distribution

Is there a way to find out if your data belongs to one or more (mixture) normal distributions? I probably could calculate what is the standard deviation of my data, but I'm not sure what else to do ...
0
votes
1answer
737 views

Standard error of normalmixEM fit?

I fitted a mixture denstiy of two gaussians two my data. I now want to calculated the standard errors of the estimates via the boot.se command of the mixtools package. My question is now, if the ...
8
votes
2answers
3k views

Why only the mean value is used in (K-means) clustering method?

In clustering methods such as K-means, the euclidean distance is the metric to use. As a result, we only calculate the mean values within each cluster. And then adjustments are made on the elements ...
15
votes
2answers
2k views

Why Expectation Maximization is important for mixture models?

There are many literature emphasize Expectation Maximization method on mixture models (Mixture of Gaussian, Hidden Markov Model, etc.). Why EM is important? EM is just a way to do optimization and is ...
4
votes
1answer
2k views

How to build a Bayesian regression model of a response that is a Gaussian mixture

Context: My response looks like a mixture model with two classes as you can see on the picture. I have a couple of predictors that perform relatively well in a linear regression (Bayesian or not). In ...
7
votes
2answers
1k views

Show that a scale mixtures of normals is a power exponential

I'm trying to show that a scale mixture of normals yields a Laplace distribution. I've gotten to the point where I have $\int N(0,\tau)\times Ga(\tau\:;\:1,\frac{\lambda^{2}}{2}) \:d\tau$ should equal ...
2
votes
1answer
605 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
6
votes
1answer
1k views

Skewness of a mixture density

I am fitting a gaussian mixture to financial data. My mixture density is given by: $f(l)=πϕ(l;μ_1,σ^2_1)+(1−π)ϕ(l;μ_2,σ_2^2)$ I calculated the skewness of the data already. Now, I want to look at ...
6
votes
2answers
816 views

Why are hidden Markov models (HMM) also called mixture models?

Why are hidden Markov models (HMM) called mixture models? What does it mix?
2
votes
1answer
2k views

Convergence of EM for Mixture of Gaussians

Is the Mixture of Gaussians model (an example of latent class analysis) gauranteed to converge on a viable solution even on Unimodal data using the Expectation Maximization algorithm to estimate the ...
2
votes
1answer
1k views

A Gaussian Mixture Model Is a Universal Approximator of Densities

When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following: A Gaussian mixture model is a universal approximator of densities, in the ...
1
vote
1answer
40 views

Understanding Loss functions in Stacked Capsule Autoencoders

I was reading Stacked Capsule Autoencoder paper published by Geoff Hinton's group last year in NIPS. While reading section 2.1 about constellation autoencoders I couldn't understand how the expression ...
3
votes
2answers
806 views

Mixture of Gaussians on Log of Data

I am practicing Mixture of Gaussians and found the below dataset snoq, which is the precipitation amounts recorded at a US region, with ...
9
votes
2answers
2k views

Applying stochastic variational inference to Bayesian Mixture of Gaussian

I am trying to implement Gaussian Mixture model with stochastic variational inference, following this paper. This is the pgm of Gaussian Mixture. According to the paper, the full algorithm of ...
8
votes
1answer
3k views

When to use LDA over GMM for clustering?

I have a dataset containing user activity with 168 dimensions, where I want to extract clusters using unsupervised learning. It is not obvious to me whether to use a topic modelling approach in Latent ...
6
votes
1answer
1k views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
6
votes
2answers
1k views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
5
votes
2answers
654 views

How to tell if a mixture of Gaussians will be multimodal

Suppose I have a mixture of Gaussians and I know the mean and variance of each separate Gaussian. How can I tell whether or not the resulting distribution will be multimodal or, more specifically, ...
4
votes
1answer
3k views

Mclust function of mclust package overfitting Gaussians

I'm using the Mclust function of the mclust package in R to fit a mixture of Gaussians model. My simulated data obviously has 3 ...
3
votes
0answers
75 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
2
votes
1answer
315 views

Conditional mean for mixture of multivariate normal distributions

If x = (x_1,x_2,...,x_n) is a vector whose components have a distribution that is a finite mixture of multivariate normals, is the expected value of x_1 still a linear function of the other components,...
1
vote
2answers
2k views

Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?

I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like: Since this was ...
3
votes
1answer
2k views

How to deal with categorical feature in a Gaussian Mixture model clustering model

I am performing clustering by Gaussian Mixture model using EM algorithm in R. U use the mclust package. My data (205 observations and 25 variables) has both ...
2
votes
1answer
760 views

GMM with Bayes decision model

Given two classes of training data (A and B), I want to fit each class' distribution using a GMM with k components, and then use Bayes Decision Model for the classification. The first step was to ...
2
votes
1answer
2k views

Derivation of M-step in EM algorithm for mixture of Gaussians

I am trying to derive the parameter estimation equations for the M-step of the expectation maximization (EM) algorithm for a mixture Gaussians when all Gaussians share the same covariance matrix $\...
1
vote
2answers
168 views

Find threshold for time varying samples

I have a df containing samples of a time varying quantity (namely, exposure to electromagnetic field generated by a GSM station), and I use the mixtools package to find a fitting mixture model (...
1
vote
1answer
994 views

How can I find mean and covariance after EM iteration on GMM algorithmm?

I have a dataset divided in 2 class(lets call x1,x2) but I don't know their mean and covariance. For each class I looked their graph and made a guess about their sub-classes, then run an EM(...