Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
5
votes
1answer
209 views

Metropolis sampling with different proposals

I implemented a metroplis sampler for a 1D gaussian mixture, the target distribution looks like this: I use a 1D normal distribution as propsal, that is each candidate is sampled from a normal ...
5
votes
1answer
187 views

how to discard values that are far from center of cluster in mixture model

I am trying to fit a bivariate cluster model with X and Y. What I would like to do is discard (make not clustered / un-grouped) that are far from the cluster center (for example $\mu$ + 2*standard ...
5
votes
2answers
188 views

Align noisy point clouds

I have a point cloud $X$ that, I suspect, is a translate of a Gaussian-corrupted version of a subset of a larger cloud $Y$, both high-dimensional ($d$ is at least 100 and ideally 10,000). What is the ...
4
votes
2answers
1k views

Mixing proportion $\pi$ in Mixtures of Gaussians

I am trying to understand a little better mixtures of Gaussians and their generative approach in general. For a mixture of Gaussians we start with this formula: $$p(x)=\sum_{k=1}^{K}\pi_{k}\cdot N(x|\...
4
votes
2answers
5k views

Gaussian Mixture Models: Maximum Likelihood Estimation or Expectation Maximization?

As far as I know the usual method for estimating the parameters in GMM is EM. However, it is also possible to use maximum likelihood. What are the differences between these two methods? Why would one ...
4
votes
2answers
715 views

Is a GMM-HMM equivalent to a no-mixture HMM enriched with more states?

I'm trying to model sequence data that has 5 hidden states. Observation data conditional to each state is gaussian except for one state for which mixture of 2 gaussians seems more appropriate. ...
4
votes
1answer
5k views

What are Gaussian Scale mixtures? And how to generate samples of Gaussian scale mixture with given scale and location parameter?

What are Gaussian scale mixture? Is it different from Gaussian mixture. What is overall location and scale parameter of given Gaussian scale mixture and how to generate a samples of given $\mu$ and $\...
4
votes
2answers
2k views

How to derive the MLE of a Gaussian mixture distribution

In my self-study, I consider a Gaussian mixture distribution: $$p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0)$$ where $p(k=1)+p(k=0)=\pi_1+\pi_0=1$. I am now asked to do three ...
4
votes
1answer
780 views

Number of Gaussian mixture components needed to approximate any distribution

I remember reading an actual proven number of components, that can approximate any distribution. Somehow I think it was 18. Can someone point me to a book/article stating something of the sort? Might'...
4
votes
2answers
3k views

What should be the covariance matrices and weights for initializing EM/GMM with kmeans?

It's typical to initialize EM for Gaussian Mixture Models using the result of kmeans clustering. However, kmeans only gives you the means (centers) of the starting GMM, but EM initialization often ...
4
votes
1answer
2k views

How to decompose a distribution with two peaks?

I'm modeling saving account's amount, whose change looks like a log-normal distribution. It means suppose $y$ is total saving account's amount; $x = \ln(y)$ is the natural log; $dx$, the daily change, ...
4
votes
1answer
3k views

Mclust function of mclust package overfitting Gaussians

I'm using the Mclust function of the mclust package in R to fit a mixture of Gaussians model. My simulated data obviously has 3 ...
4
votes
2answers
327 views

ANOVA-type test but with known population variance of each group

I have a set of $N$ samples $s_{i}$, each one sampled from a normal distribution with standard deviations $\sigma_i$, which are known. I would like to know if the distributions have the same mean. I ...
4
votes
4answers
221 views

Finite mixture models - Basic understanding

I have been reading lecture slides about Dirichlet Process. In page 22, there is a picture about the following finite mixture model. $$\phi _{k}\sim H\\ \pi \sim Dirichlet(\alpha /K,\dots,\alpha /K)\...
4
votes
1answer
61 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
4
votes
1answer
355 views

EM algorithm gaussian mixtures- derivation

I'm trying to make sense of a derivation I'm following from the lecture notes of Stanford's ML course. Specifically the notes are here: http://cs229.stanford.edu/notes/cs229-notes8.pdf I'm ...
4
votes
1answer
818 views

What is the median of an equally weighted mixture of two Normal Distributions?

Suppose men's heights follow a normal distribution $X \sim \mathcal{N}(\mu_1,\sigma_1^2)$ and women's heights follow a normal distribution $Y \sim \mathcal{N}(\mu_2,\sigma_2^2)$. How can I find the ...
4
votes
1answer
633 views

gaussian mixture model - approximate a matrix

I have a similarity matrix M - the value M(i,j) indicates the similarity between two elements i and j. I want to approximate that matrix using a Gaussian Mixture model or I want to cluster that ...
4
votes
1answer
4k views

anomaly detection with gaussian mixture models

I am new to the topic, and I am trying to understand how it is possible to perform anomaly detection by using gaussian mixture models. Could you please give me some hints about literature on the topic?...
4
votes
2answers
323 views

Gaussian Mixture Model parameters from density

How do I estimate parameters of subpopulations in a 1D gaussian mixture model when I already have density (measured on a grid) of the mixture? All the algorithms I can find (like the well-known EM ...
4
votes
2answers
1k views

Gaussian Mixture for detecting outliers

I'm trying to make a simple outlier detection program that is able to correctly, or almost correctly, identify values in a data set that could be potential outliers because they don't fall in the ...
4
votes
1answer
611 views

Machine Learning : Classification algorithm for very high dimensional data which is uniquely definable in a very small sub-space

I am new to machine learning, so forgive me if i am doing something absolutely absurd. I have a classification task (~100 classes) and have about 2 million training data points in a 2000 dimensional ...
4
votes
0answers
32 views

Implementation of EM algorithm confusion

Here EM algorithm manually implemented, there's a question of the implementation in R of the EM algorithm for 2 mixed gaussians. The answer has a supposedly correct implementation. However, don't the ...
4
votes
0answers
1k views

Finite mixture models with bounded data

I am trying to fit a finite mixture model to a dependent variable which is bounded (practically) between -0.594 and 1 (theoretically, the latent variable is bounded between -Inf - 1). The data are ...
4
votes
0answers
3k views

Ftting a mixture of two Gaussians

I want to fit a mixture of two gaussian densities to my financial data. The data can be found here: http://uploadeasy.net/upload/2a7mw.rar the variable is called dat. The probability density of a ...
3
votes
1answer
3k views

Fit mixture of distributions to your time-series data in R

I have time-series data containing 1440 observations and the plot of the data is I want to fit the Gaussian Mixture Models (GMM) to the above plot, and for the same I am using Mclust function of ...
3
votes
1answer
1k views

How to build a Bayesian regression model of a response that is a Gaussian mixture

Context: My response looks like a mixture model with two classes as you can see on the picture. I have a couple of predictors that perform relatively well in a linear regression (Bayesian or not). In ...
3
votes
2answers
406 views

Distribution of the exponential of a mixture?

Suppose that $X$ is distributed as a finite mixture of normals $$\sum_{j=1}^k w_j \phi(x;\mu_j,\sigma_j^2).$$ Is $\exp(X)$ distributed as a finite mixture of log-normal distributions?
3
votes
2answers
368 views

Gaussian mixture model - does an improper uniform prior give a proper posterior?

We draw $n$ i.i.d. points $x_1 , x_2 , ..., x_n$ from the following Gaussian mixture: $$p(x|\mu_1,\mu_2) = \frac{1}{2} \text{N} (x|\mu_1,1) + \frac{1}{2} \text{N} (x|\mu_2,1).$$ The prior is the ...
3
votes
1answer
2k views

MLE of the mixture parameter in mixing two normal densities

Imagine that we have mixture of two normal distributions with mixture parameter $\theta$: $$p(y_i|\theta) = \theta\phi(y_i;\mu_1, \sigma_1^2) + (1 - \theta)\phi(y_i; \mu_2, \sigma_2^2)$$ Assume that ...
3
votes
1answer
794 views

Median of mixture of two Gaussian distributions with equal weights

I am given a population $P$ that is equally divided into subsets $A$ and $B$. I know that a property $H$ of the population $P$ is normally distributed with mean $\mu_1$ and variance $\sigma_1^2$ for ...
3
votes
2answers
3k views

How to use Kullback-leibler divergence if mean and standard deviation of of two Gaussian Distribution is provided?

With Apache Spark MLLib library I am trying to find Clusters within a dataset by using Gaussian Mixture Model (number cluster =3) . Now it returns 3 different values of mean and standard deviation. I ...
3
votes
1answer
689 views

Is there a numerical solution to a mixture model of two normal distributions?

I'm building a mixture model with the two normal distributions $\mathcal{N}(\mu_1,\sigma_{1}^{2})$ and $\mathcal{N}(\mu_2,\sigma_{2}^{2})$. So, the density function is $$ f(x) = p_1 N(x; \mu_1, \...
3
votes
1answer
140 views

How to cluster parts of broken line made of points?

I am studying clustering techniques and i am pretty new at this topic. Here is my problem: I created a 5 lines which are made of points. This lines are supposed to be continuous and they look like ...
3
votes
2answers
3k views

Significant difference between time series - Can I do this?

I'd like to know whether the solution proposed below is valid/acceptable and any justification available. We have two biological conditions, and for each condition we measured 3 time series, so at ...
3
votes
1answer
521 views

conjugate prior for (multivariate) Gaussian mixtures (with known mean and covariance)?

Say I have a (multivariate) Gaussian mixture model $$p(x)=\sum_k\pi_iN(\mu_i,\Sigma_i),$$ of which the $\boldsymbol\mu$ and $\boldsymbol\Sigma$ are known, so the likelihood function of the ...
3
votes
1answer
546 views

Is there anything wrong with performing EM clustering on PCA output?

I am trying to separate my dataset into meaningful clusters. I have tried k-means, hierarchical and EM clustering (fitting a gaussian mixture model using EM algorithm, using the EMCluster R package) ...
3
votes
1answer
669 views

Does a Gaussian mixture model always imply a within-class multivariate normal probability distribution?

If I use a latent profile analysis (Gaussian Mixture Model) to model my observed multivariate probability distribution as a mixture (K-classes) of conditionally-independent normal pdfs, does this ...
3
votes
1answer
626 views

EM algorithm with a component for outliers

i have a vector of measurements from one to three classes, which can be modeled by gaussian distributions. There are some outliers in the data. I use the EM algorithm to learn the parameters of the ...
3
votes
1answer
327 views

Are growth mixture models just Gaussian mixtures applied to coefficients of polynomials fitted to time-series data?

Am I understanding correctly that growth mixture model is just Gaussian mixtures applied to coefficients of polynomials fitted to the time-series data? For example, we have 1000 cases, with 3 ...
3
votes
1answer
82 views

Sampling posterior of empty cluster in GMM and Gibbs

Consider performing inference via a standard Gibbs sampler for a standard Gaussian Mixture Model (GMM) with $k$ components that are Gaussians $$\mathcal{N}(\mu_{k}, \sigma^{2}_{k})$$ where we assume ...
3
votes
1answer
762 views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
3
votes
1answer
661 views

How to reestimate GMMs in a HMM-GMM

Context: Automatic Speech Recognition I understand the training of a pure HMM with Baum-Welch: Expectation step compute $\gamma_t(i) = P(q_t=i |O,\lambda)$ //p(passing state $i$ at frame $...
3
votes
1answer
1k views

PyMC for Categorical Latent Model

I'm learning PyMC and am trying to fit a simple categorical mixture model but the sampling estimates don't converge to the true values. I'm wondering if I've specified the model incorrectly or am ...
3
votes
4answers
1k views

When does the EM for Gaussian mixture model has one of the Gaussian diminish to exactly one point and have zero variance?

I had implemented the EM algorithm for mixture models as follows: For the E-step I compute the soft-counts of assigning each point $x^{(t)} \in Data_n$ to an individual cluster $j \in \{1, ..., K \}$ ...
3
votes
1answer
664 views

Mixture of Gaussians on Log of Data

I am practicing Mixture of Gaussians and found the below dataset snoq, which is the precipitation amounts recorded at a US region, with ...
3
votes
0answers
36 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
3
votes
0answers
50 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
3
votes
0answers
33 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
3
votes
0answers
695 views

Correct number of components in GMM according to BIC and AIC plots

I have applied GMM(Gaussian Mixture Model) to my data set and I have plotted the resulting BIC(Bayesian Information Criterion) and AIC(Akaike Information Criterion) for different number of components. ...