Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

4
votes
2answers
1k views

Gaussian Mixture for detecting outliers

I'm trying to make a simple outlier detection program that is able to correctly, or almost correctly, identify values in a data set that could be potential outliers because they don't fall in the ...
0
votes
1answer
797 views

Negative loss while training Gaussian Mixture Density Networks

In classification problems, the usual negative log-likelihood loss function $L(\theta)=\sum_{i=1}^N -\log(p(y_i|x_i,\theta))$ is always non-negative, since the $y_i$'s are discrete random variables ...
2
votes
1answer
919 views

comparing gaussian mixture models with and without regularization

I am using Gaussian mixture models for clustering a bunch of data sets. On some data sets a regularization value is often used/suggested (see the MATLAB example for Fisher's Iris data which sets ...
0
votes
1answer
345 views

Fitting Gaussian mixture model to binary time data

I have data about the dates/times a user logs in and out of a system. I am trying to model the likelihood a user will be logged into the system at any particular time of day. I want to fit a Gaussian ...
0
votes
2answers
948 views

Spark MLLib Gaussian Mixture Model feature or bug

Is this expected from Gaussian Mixture Model? Given a perfectly homogenous dataset, the cluster center is not exactly the same as the data point? ...
5
votes
5answers
474 views

Choosing a model for my unsupervised machine learning problem

I need to choose a model for unsupervised machine learning problem. There are 4 clusters in 3D space. These are my requirements: I will run the same model multiple times with different training data (...
1
vote
0answers
36 views

Approximating a 3D spline surface by a weighted sum of gaussians

I have spatial data(2D) with some quantity associated with each point - basically 3D data. I want to model the quantity distribution in the space and then use N clusters as a compact representation. ...
2
votes
0answers
62 views

Distribution of the initializing set at K-means++

There is a well-known modification of the initializing step of K-means, named K-means++. It chooses cluster centers with probability proportional to its squared distance from the point's closest ...
1
vote
1answer
254 views

Bimodal univariate distributions are always indicative of a mixture of two random variables. Is this correct? [duplicate]

Say I see a bimodal distribution like this (with the domain, or random variable, $Z$): Does that instantly mean that I am seeing not a distribution of one independent random variable $Z$, but ...
1
vote
0answers
15 views

how can I integrate probabilities from two GMMs?

Without going into the details of why exactly I must do this, I have four GMMs (two sets for two classes), and I need to integrate their predictions. Two of them are trained on two classes from ...
1
vote
0answers
2k views

Computing covariance matrix and mean in python for a Gaussian Mixture Model

I am studying Bishop's PRML book and trying to implement Gaussian Mixture Model from scratch in python. So I have prepared a synthetic dataset which is divided into 2 classes using the following ...
0
votes
1answer
241 views

Does a mixture model need to sum or integrate to $1$?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean $\mu$ and variance $\...
2
votes
1answer
923 views

How to deal with categorical feature in a Gaussian Mixture model clustering model

I am performing clustering by Gaussian Mixture model using EM algorithm in R. U use the mclust package. My data (205 observations and 25 variables) has both ...
1
vote
0answers
154 views

Improve gaussian mixture model performance

I have a data set of 10-dimensional cell measurements for leukemia. The data points are unlabeled and the task is to find the ratio of pathological measurements w.r.t. the rest of the sample. In other ...
1
vote
1answer
1k views

Assessing Gaussian mixture distribution by cross validation

I have a 10 dimensional random vector that I'm modelling with GMMs. I want to estimate the best number of mixtures ($K$) for my data via the following method: Divide the data to train (90%) and test (...
1
vote
2answers
749 views

Manually generate random sample in Gaussian mixture model

I want to generate (manually) a random sample in the Gaussian mixture model: $$f_{\theta}(x) = \sum_{k = 1}^{K}\pi_k f_{\mathcal N(\mu_k, \sigma^2_k)}(x)$$ Here is my work: ...
1
vote
0answers
155 views

Gassuian mixture model visualization

Let $x\in\mathbb{R}^n$ be a GMM-distributed vector with $K$ components. In the setting where we consider diagonal covariances, we have $P1=K\cdot (n+n)+K$, i.e. linear in $n$ and $K$, and $P_2=K(n^2/2+...
2
votes
1answer
2k views

Sampling from a multivariate Gaussian mixture model [duplicate]

How can I generate multi-dimensional data from a (estimated) Gaussian mixture pdf? In general, what would be ways to generate multi-dimensional data from a pdf? I read rejection sampling can be used, ...
1
vote
0answers
78 views

Which parameters need to be initialized random for gaussian mixture hidden markov model?

So, if I model observation probability for a given hidden state according to a multivariate gaussian mixture model, then which parameters need to be initialized random to perform parameter re-...
1
vote
1answer
592 views

How can I find mean and covariance after EM iteration on GMM algorithmm?

I have a dataset divided in 2 class(lets call x1,x2) but I don't know their mean and covariance. For each class I looked their graph and made a guess about their sub-classes, then run an EM(...
2
votes
0answers
385 views

Approximating Uniform Distribution with Mixture of Gaussians

Let $T$ be a compact, connected, proper subset of $\mathbb{R}^3:\quad T \subset \mathbb{R}^3$. Further let $\left\{ \boldsymbol{\mu}_i \right\}_{i=1}^n$ be a given finite set of $n$ point in $T$: $$ \...
0
votes
0answers
102 views

Two variables in Multivariate Gaussian pdf

I have problem with computing multivariate gaussian probability density function(pdf) value. As I found the equation is, $$p(\textbf{x}|\mu, \Sigma) = \frac{1}{(2\pi )^{\frac{n}{2}} |\Sigma|^{\frac{1}...
0
votes
1answer
122 views

How to determine cluster on EM-Gaussian Mixture clustering with 2 or more variables

I'm trying to compute EM Gaussian Mixture clustering algorithm. As I found in Bishop(2009), it explained the algorithm. Which is we have E-step and M-step in the iteration process. And we could ...
2
votes
0answers
520 views

Programming a mixture of a Gamma with a Normal distribution using R

I have some data x in R which seems to be a mixture of a Gamma and Normal distribution. Therefore I'd like to model this as a mixture model consisting of said distributions, but I don't know how to ...
2
votes
1answer
285 views

Need intuition - how do they simplify the Q function for gaussian mixture EM?

Background - I'm trying to follow section 7.2.4 in this EM tutorial. Basically the setup is I have a vector with 10 points $x$, and each of them can be assigned ($y$) to either Gaussian 1 or to ...
14
votes
2answers
1k views

Why Expectation Maximization is important for mixture models?

There are many literature emphasize Expectation Maximization method on mixture models (Mixture of Gaussian, Hidden Markov Model, etc.). Why EM is important? EM is just a way to do optimization and is ...
2
votes
1answer
211 views

model selection, mixture of Gaussians

I have data and I want to decide whether it comes from 5-modal-normal distribution or 2-modal-normal distribution. In other words I want to check if it has 2 peaks or 5. I can estimate the $\mu$ and $\...
8
votes
2answers
6k views

When to use Gaussian mixture model?

I am new to using GMMs. I was not able to find any appropriate help online. Could anyone please provide me right resource on "How to decide if using GMM fits to my problem?" or in case of ...
0
votes
1answer
47 views

How can I estimate the probability that my observed data come from a bimodal population?

I have a data set representing the abundance of a protein in a population of cells. Based on our understanding of the biology behind this, I expect there to be two subpopulations - one in which this ...
1
vote
1answer
523 views

Understanding hidden markov model, and how it is applied in speech recognition

I have for some some time tried to understand how this hidden markov model (hmm) works, and have found a lot of tutorials/papers on it which make use of the same examples/principles of explaining the ...
3
votes
1answer
1k views

How to build a Bayesian regression model of a response that is a Gaussian mixture

Context: My response looks like a mixture model with two classes as you can see on the picture. I have a couple of predictors that perform relatively well in a linear regression (Bayesian or not). In ...
0
votes
1answer
330 views

How could I find the equation for joint probability function?

if two processes X and Y are marginally univariate gaussian, and given the fact we know all the parameters for those processes, then how could I find the joint probability function, p(X,Y)? if those ...
1
vote
0answers
26 views

Mixture models - am I on the right path?

We have 2 chemical parameters measured for material 1 and 2. We are interested in finding out the probability of an unknown sample being the mixture of the two reference materials. Also determination ...
5
votes
1answer
190 views

Metropolis sampling with different proposals

I implemented a metroplis sampler for a 1D gaussian mixture, the target distribution looks like this: I use a 1D normal distribution as propsal, that is each candidate is sampled from a normal ...
2
votes
1answer
216 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
3
votes
1answer
218 views

Universal Approximation Capabilities of Mixture Models

I am looking for two reference incl. proofs showing 1) that a discrete Mixture of Gaussians can asymptotically approximate any (well behaved) continuous density 2) that a discrete Mixture of ...
2
votes
1answer
2k views

Differences linear discriminant analysis and Gaussian mixture model

I know that there are topics about this question but in my view, the answers are not clear enough. I don't understand the main difference between Linear Discriminant Analysis (LDA) and Gaussian ...
0
votes
1answer
122 views

Why is my Gaussian mixture plot reduced proportional in size compared to their univariate normal pdf

I have created a multiples Gaussian distributions using this codes in MATLAB: ...
1
vote
0answers
611 views

Mixture of bivariate normal distribution in R

I would have to define, in R, a mixture of a number of bivariate normal distributions like that: a strategy would be to define the single pieces of the expressions, for example: ...
0
votes
0answers
128 views

upper bound on number of components of GMM

I want to fit GMM to a data set with N data points in D dimensions. I am using the full GMM in MATLAB, i.e., each component has a complete covariance matrix (as opposed to diagonal covaraices which ...
1
vote
1answer
121 views

Learn parameters for truncated Gaussian

I would like to learn the parameters for a truncated gaussian like this one. I'm using this formula for the probability density $f(x | \mu, \sigma^2) = \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \...
1
vote
1answer
60 views

Intuition around weighted normal distributions standard deviations

I am looking at a set of normal distributed data and trying to figure out why my intuition is wrong here. If I have multiple normal distributions $N_m(M_m, S_m)$ the literature tells me that I can ...
6
votes
2answers
779 views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
5
votes
2answers
472 views

Why are hidden Markov models (HMM) also called mixture models?

Why are hidden Markov models (HMM) called mixture models? What does it mix?
2
votes
1answer
97 views

Useful separation value in a mixture distribution

Assume we have a distribution that is the mixture of two normal distributions. The pdf of the overall distributions and their single parts may look like the following. In black, the combined ...
1
vote
0answers
112 views

Creating observations to test gaussian mixture HMMs [closed]

I have a Gaussian mixture HMM $\lambda = (\pi, A, B)$ in which $\pi$ is the starting state probabilities, $A$ is the state transition matrix, and $B$ is the emissions matrix. I fit a GMM into the ...
2
votes
1answer
160 views

Gaussian mixture distribution confusion

Have some confusion on Gaussian mixture distribution model (reference is from the book of Pattern Recognition and Machine Learning). My confusion is how below formula works? $\sum_kz_k=1$ My thought ...
1
vote
1answer
91 views

What does 'vector-valued' mean?

What is the difference of a feature vector and a 'vector-valued observation' as described here? The term 'vector-valued' is used in the following context: "Most state-of-the-art [Automatic Speech ...
0
votes
0answers
35 views

System of Gaussian equations

Let $N(x\ |\ \mu,\sigma^2)$ be the pdf of a normal random variable with mean $\mu$ and variance $\sigma^2$. Question: Given $n$ data points $(x_i,y_i)_{i=1,\dots,n}$, compute $\{w_i\}_i$,$\{\sigma_i^2\...
9
votes
2answers
2k views

Applying stochastic variational inference to Bayesian Mixture of Gaussian

I am trying to implement Gaussian Mixture model with stochastic variational inference, following this paper. This is the pgm of Gaussian Mixture. According to the paper, the full algorithm of ...