Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
6
votes
1answer
3k views

Ftting a mixture of two Gaussians

I want to fit a mixture of two gaussian densities to my financial data. The data can be found here: http://uploadeasy.net/upload/2a7mw.rar the variable is called dat. The probability density of a ...
6
votes
2answers
1k views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
5
votes
2answers
1k views

Is a GMM-HMM equivalent to a no-mixture HMM enriched with more states?

I'm trying to model sequence data that has 5 hidden states. Observation data conditional to each state is gaussian except for one state for which mixture of 2 gaussians seems more appropriate. ...
5
votes
1answer
3k views

Is it important to make a feature scaling before using Gaussian Mixture Model?

Is it important to make a feature scaling before using Gaussian Mixture Model? and why is it important while we are using probability in getting our clusters's parameters (mean and covariance matrix). ...
5
votes
1answer
949 views

Number of Gaussian mixture components needed to approximate any distribution

I remember reading an actual proven number of components, that can approximate any distribution. Somehow I think it was 18. Can someone point me to a book/article stating something of the sort? Might'...
5
votes
5answers
741 views

Choosing a model for my unsupervised machine learning problem

I need to choose a model for unsupervised machine learning problem. There are 4 clusters in 3D space. These are my requirements: I will run the same model multiple times with different training data (...
5
votes
2answers
3k views

Sum of Gaussian mixture and Gaussian scale mixture

What will be the distribution of the sum of two independent random variables, say $X$ and $Y$, when $X$ has a Gaussian mixture distribution (when we take Gaussian distribution with different location ...
5
votes
1answer
3k views

How to decompose a distribution with two peaks?

I'm modeling saving account's amount, whose change looks like a log-normal distribution. It means suppose $y$ is total saving account's amount; $x = \ln(y)$ is the natural log; $dx$, the daily change, ...
5
votes
1answer
2k views

MLE of the mixture parameter in mixing two normal densities

Imagine that we have mixture of two normal distributions with mixture parameter $\theta$: $$p(y_i|\theta) = \theta\phi(y_i;\mu_1, \sigma_1^2) + (1 - \theta)\phi(y_i; \mu_2, \sigma_2^2)$$ Assume that ...
5
votes
1answer
150 views

A proof that the median is a nonlinear statistical functional

This question is with reference to the top answer (by @StephanKolassa) to this question. Let $F$ and $G$ be CDFs and define $$H(x)=aF(x)+(1-a)G(x)$$ with $a\in [0, 1]$. Now suppose $F$ and $G$ are ...
5
votes
2answers
4k views

Clustering methods for unknown number of clusters

Matrix $X=[x_1,...,x_i,...,x_N]$ is a data-set containing $N$ data-points that each data-point $x_i$ is a vector of $D$ dimensions. Each dimension is a feature. The number of clusters ($K$) is unknown....
5
votes
2answers
460 views

Grouping similar subjects based on slopes

I am studying the responses of 500 subjects to temperature increase using a linear regression on each degree of temperature (from 10°C to 28°C). Thus, it was possible for me to compute the intercept ...
5
votes
2answers
681 views

the approximation power of Gaussian mixture models?

What are the probability density functions that GMM can approximate? a reference in appreciated about this.
5
votes
1answer
340 views

Metropolis sampling with different proposals

I implemented a metroplis sampler for a 1D gaussian mixture, the target distribution looks like this: I use a 1D normal distribution as propsal, that is each candidate is sampled from a normal ...
5
votes
1answer
2k views

Selecting the number of mixtures / hidden states / latent variables

My question is regarding Gaussian Mixture models, Hidden Markov models (HMM) or any type of clustering or latent variable model, for which we can devise a likelihood function. Specifically, I train a ...
5
votes
1answer
873 views

gaussian mixture model - approximate a matrix

I have a similarity matrix M - the value M(i,j) indicates the similarity between two elements i and j. I want to approximate that matrix using a Gaussian Mixture model or I want to cluster that ...
5
votes
1answer
452 views

Derivation of maximum likelihood for a Gaussian mixture model

I'm working my way through the derivation of EM in Bishop (p. 435). I'm stuck trying to derive to MLE for $\mu_k$ for the gaussian mixture model. Basically I get an extra sum in the numerator. For ...
5
votes
1answer
6k views

anomaly detection with gaussian mixture models

I am new to the topic, and I am trying to understand how it is possible to perform anomaly detection by using gaussian mixture models. Could you please give me some hints about literature on the topic?...
5
votes
1answer
1k views

PyMC for Categorical Latent Model

I'm learning PyMC and am trying to fit a simple categorical mixture model but the sampling estimates don't converge to the true values. I'm wondering if I've specified the model incorrectly or am ...
5
votes
2answers
422 views

Gaussian Mixture Model parameters from density

How do I estimate parameters of subpopulations in a 1D gaussian mixture model when I already have density (measured on a grid) of the mixture? All the algorithms I can find (like the well-known EM ...
5
votes
1answer
192 views

how to discard values that are far from center of cluster in mixture model

I am trying to fit a bivariate cluster model with X and Y. What I would like to do is discard (make not clustered / un-grouped) that are far from the cluster center (for example $\mu$ + 2*standard ...
5
votes
2answers
230 views

Align noisy point clouds

I have a point cloud $X$ that, I suspect, is a translate of a Gaussian-corrupted version of a subset of a larger cloud $Y$, both high-dimensional ($d$ is at least 100 and ideally 10,000). What is the ...
5
votes
2answers
2k views

Maximum likelihood estimation for Gaussian mixture

When doing maximum likelihood (ML) inference on a Gaussian mixture model (GMM), Bishop notes in PRML that if there is more than one mixture component in the GMM and the mean of one Gaussian collapses ...
4
votes
2answers
2k views

Mixing proportion $\pi$ in Mixtures of Gaussians

I am trying to understand a little better mixtures of Gaussians and their generative approach in general. For a mixture of Gaussians we start with this formula: $$p(x)=\sum_{k=1}^{K}\pi_{k}\cdot N(x|\...
4
votes
3answers
3k views

What's the difference between Multivariate Gaussian and Mixture of Gaussians?

What's the difference between Multivariate Gaussian and Mixture of Gaussians? If I have a Multivariate Gaussian and making all the data into ONE vector, is that a Mixture of Gaussians in 1 dimension?...
4
votes
2answers
8k views

Standard deviation for weighted sum of normal distributions

I have 2 normally distributed random variable $H_0$ and $H_1$, which are combined to give the weighted distribution $H$ as follows: $H_0 \sim N(\mu_0, \sigma_0)$ $H_1 \sim N(\mu_1, \sigma_1)$ $$f_H ...
4
votes
1answer
2k views

How to build a Bayesian regression model of a response that is a Gaussian mixture

Context: My response looks like a mixture model with two classes as you can see on the picture. I have a couple of predictors that perform relatively well in a linear regression (Bayesian or not). In ...
4
votes
2answers
4k views

How to derive the MLE of a Gaussian mixture distribution

In my self-study, I consider a Gaussian mixture distribution: $$p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0)$$ where $p(k=1)+p(k=0)=\pi_1+\pi_0=1$. I am now asked to do three ...
4
votes
1answer
3k views

Convergence of EM for Mixture of Gaussians

Is the Mixture of Gaussians model (an example of latent class analysis) gauranteed to converge on a viable solution even on Unimodal data using the Expectation Maximization algorithm to estimate the ...
4
votes
1answer
68 views

Given two normal populations,, classifying a given data point

I have two normal populations S1 and S2, where S1 ~ N (μ1, σ1) and S2 ~ N (μ2, σ2) respectively. The populations are independent of each other and a data point X has to be either from S1 or from S2. ...
4
votes
2answers
663 views

Gaussian mixture model - does an improper uniform prior give a proper posterior?

We draw $n$ i.i.d. points $x_1 , x_2 , ..., x_n$ from the following Gaussian mixture: $$p(x|\mu_1,\mu_2) = \frac{1}{2} \text{N} (x|\mu_1,1) + \frac{1}{2} \text{N} (x|\mu_2,1).$$ The prior is the ...
4
votes
2answers
6k views

How to use Kullback-leibler divergence if mean and standard deviation of of two Gaussian Distribution is provided?

With Apache Spark MLLib library I am trying to find Clusters within a dataset by using Gaussian Mixture Model (number cluster =3) . Now it returns 3 different values of mean and standard deviation. I ...
4
votes
2answers
1k views

Which metric is used in the EM algorithm for GMM training ?

My question concerns the expectation-maximisation algorithm used to estimate the hyper-parameters of a Gaussian mixture model in z multivariate setup. I understand that the EM algorithm uses the ...
4
votes
1answer
1k views

Median of mixture of two Gaussian distributions with equal weights

I am given a population $P$ that is equally divided into subsets $A$ and $B$. I know that a property $H$ of the population $P$ is normally distributed with mean $\mu_1$ and variance $\sigma_1^2$ for ...
4
votes
2answers
5k views

What should be the covariance matrices and weights for initializing EM/GMM with kmeans?

It's typical to initialize EM for Gaussian Mixture Models using the result of kmeans clustering. However, kmeans only gives you the means (centers) of the starting GMM, but EM initialization often ...
4
votes
4answers
285 views

Finite mixture models - Basic understanding

I have been reading lecture slides about Dirichlet Process. In page 22, there is a picture about the following finite mixture model. $$\phi _{k}\sim H\\ \pi \sim Dirichlet(\alpha /K,\dots,\alpha /K)\...
4
votes
1answer
1k views

Is there a numerical solution to a mixture model of two normal distributions?

I'm building a mixture model with the two normal distributions $\mathcal{N}(\mu_1,\sigma_{1}^{2})$ and $\mathcal{N}(\mu_2,\sigma_{2}^{2})$. So, the density function is $$ f(x) = p_1 N(x; \mu_1, \...
4
votes
1answer
89 views

Bayesian mixture model joint posterior

I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
4
votes
1answer
115 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
4
votes
1answer
3k views

Mclust function of mclust package overfitting Gaussians

I'm using the Mclust function of the mclust package in R to fit a mixture of Gaussians model. My simulated data obviously has 3 ...
4
votes
2answers
420 views

ANOVA-type test but with known population variance of each group

I have a set of $N$ samples $s_{i}$, each one sampled from a normal distribution with standard deviations $\sigma_i$, which are known. I would like to know if the distributions have the same mean. I ...
4
votes
1answer
303 views

What is the marginal posterior distribution?

Based on this question: How to build a Bayesian regression model of a response that is a Gaussian mixture Consider the mixture of normal, $$y_j\sim (N(0,\sigma_1))^{\pi}(N(0, \sigma_2))^{1-\pi}, j=1,...
4
votes
1answer
591 views

EM algorithm gaussian mixtures- derivation

I'm trying to make sense of a derivation I'm following from the lecture notes of Stanford's ML course. Specifically the notes are here: http://cs229.stanford.edu/notes/cs229-notes8.pdf I'm ...
4
votes
1answer
984 views

What is the median of an equally weighted mixture of two Normal Distributions?

Suppose men's heights follow a normal distribution $X \sim \mathcal{N}(\mu_1,\sigma_1^2)$ and women's heights follow a normal distribution $Y \sim \mathcal{N}(\mu_2,\sigma_2^2)$. How can I find the ...
4
votes
1answer
258 views

Are Neural Networks Mixture Models?

To my understanding, Gaussian Mixture models are a set of parameterized gaussian distributions that collectively describe an entire, aggregate distribution. ^ from McGonagle et al Also to my ...
4
votes
1answer
969 views

Is there anything wrong with performing EM clustering on PCA output?

I am trying to separate my dataset into meaningful clusters. I have tried k-means, hierarchical and EM clustering (fitting a gaussian mixture model using EM algorithm, using the EMCluster R package) ...
4
votes
4answers
2k views

When does the EM for Gaussian mixture model has one of the Gaussian diminish to exactly one point and have zero variance?

I had implemented the EM algorithm for mixture models as follows: For the E-step I compute the soft-counts of assigning each point $x^{(t)} \in Data_n$ to an individual cluster $j \in \{1, ..., K \}$ ...
4
votes
1answer
938 views

Meaning of Gaussian mixture weights?

other posts with similar title do not actually ask what's in their title, so I ask here: What is the meaning of the weights in the Gaussian Mixture Model (GMM)? Does the GMM weight more heavily to ...
4
votes
1answer
146 views

Sampling posterior of empty cluster in GMM and Gibbs

Consider performing inference via a standard Gibbs sampler for a standard Gaussian Mixture Model (GMM) with $k$ components that are Gaussians $$\mathcal{N}(\mu_{k}, \sigma^{2}_{k})$$ where we assume ...
4
votes
1answer
905 views

Machine Learning : Classification algorithm for very high dimensional data which is uniquely definable in a very small sub-space

I am new to machine learning, so forgive me if i am doing something absolutely absurd. I have a classification task (~100 classes) and have about 2 million training data points in a 2000 dimensional ...

1
2
3 4 5
12