Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

1
vote
1answer
201 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
1
vote
1answer
192 views

Conditional mean for mixture of multivariate normal distributions

If x = (x_1,x_2,...,x_n) is a vector whose components have a distribution that is a finite mixture of multivariate normals, is the expected value of x_1 still a linear function of the other components,...
1
vote
2answers
152 views

Meaning of Gaussian mixture model parameters

I came across this question from a tutorial: Suppose we have observations $x_1$ , $x_2$ , $\ldots$, $x_n$ of a continuous r.v. $X$ known to be drawn from a “mixture” of $k$ Gaussian distributions. ...
1
vote
1answer
535 views

Estimating truncation point in Gaussian mixture

I have data modeled as a mixture of two Gaussian distributions. The data is "clipped" i.e., there is data only for values greater than a threshold $t$, even though it is feasible for data to exist in ...
0
votes
0answers
55 views

Reducing the number of Gaussians in a Gaussian Mixture Model

I build a kernel density estimation (KDE) of Gaussian kernels. I have many samples, but the distribution is not too complicated. I think it should be possible to approximate the resulting KDE by a ...
2
votes
0answers
35 views

Multimodality of mixtures of more than two Normal distributions

Let $$\phi(x;\mu,\sigma) = \frac{1}{\sigma \sqrt{2\pi}} \exp \left(- \frac{(x-\mu)^2}{2\sigma^2}\right)$$ denote the Gaussian density function ($\sigma > 0$). Let $$f(x) = \sum_{i=1}^N p_i \...
1
vote
0answers
72 views

Interpretation of plots for outlier detection in healthcare

Christy et al. propose cluster-based approach to outlier detection as part of the preprocessing step. However, I don't think the plots are very interpretable. The authors use the R mclustbic function ...
0
votes
0answers
75 views

how to model hourly wind speed data

I am trying to forecast hourly wind speed (HWS) data in Trinidad and Tobago and I have read in the literature that "Direct application of stochastic models (ARMA & ARIMA models) to HWS series is ...
0
votes
1answer
199 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
2
votes
0answers
297 views

GMM EM algorithm complexity per iteration

I was fitting GMM clusters with diagonal covariance on my data using EM with $n$ (=5e6) points, each having $m$ (=160) ...
0
votes
1answer
177 views

JAGS mixture models with exogenous regressors

This is my first post,I hope this is the right forum for such a question and I formulate it correctly. I am working with a time series data set where the response y seems to follow a mixture of two ...
1
vote
0answers
52 views

Mean and variance of a vector $W = (X', Y')'$

$(X_i,Y_i)$, $i = 1,2, ... ,n$, is a random sample from a bivariate normal distribution, with means $\mu_x$ and $\mu_y$, variances $\sigma_x$ and $\sigma_y$, and correlation $\rho$. How do I formally ...
1
vote
1answer
32 views

Mixture modelling of data with measurement uncertainty

I have a dataset that consists of a population radiometric ages (300>n>600). A dataset can have ages can range on the order of billions of years. Each age measurement has an associated uncertainty ...
0
votes
1answer
245 views

For Multivariate Gaussian Mixture Models, What will happen if all mixing probabilities are equal? [closed]

$$p(\boldsymbol{x}) =\sum_{k=1}^K \pi_k \mathcal{N}(\boldsymbol{x}|\boldsymbol{\mu}_k, \boldsymbol{\Sigma}_k) $$ This is the formula for MGMM, where $\pi_k$ is the mixing probability. I am very ...
2
votes
0answers
67 views

Gaussian Mixture Division

In the study of probabilistic graphical models (PGMs), the loopy belief update propagation (LBUP) message passing algorithm requires the division of unnormalised probability distributions. If the ...
0
votes
2answers
1k views

How to extract components of Gaussian mixture?

I'm trying to model a dataset as a mixture of two Gaussian distributions in MATLAB and find the Bhattacharyya distance between the two. Using MATLAB's fitgmdist ...
2
votes
2answers
207 views

This is Gaussian mixture model?

Here is a problem that I am looking at. Is this model really commonly known as a Gaussian mixture model (the one often appears as an illustration of EM algorithm)? I am confused because Gaussian ...
0
votes
1answer
83 views

Finding a Dominant Cluster

Super-basic question here: I'm looking for a way to find the dominant cluster of a set of clusters (as in the first image): This is not what I get when I run a Gaussian Mixture model with one ...
1
vote
2answers
856 views

Mixed data in Gaussian Mixture Models

Is it possible to use a dataset with mixed variables such as continuous, ordered, and categorical variables and cluster the data using the Gaussian Mixed Model with EM algorithm. I cannot find ...
0
votes
1answer
454 views

Difference between class conditional distribution and likelihood in the context of Mixture of Gaussians?

I am reading some Mahchine learning lecture notes, and the writer is introducing Maximum Likelihood (ML) method of parameter estimation of $\theta$ as $\text{argmax}_{\theta}Pr_{\theta}(x|H)$ where $H$...
0
votes
1answer
96 views

Three component mixture model for element concentration using mixtools in R

As an update to a previously posed question, Fitting a mixture distribution for two approximately normally distributed populations using mixtools in R , I have now fit a three component mixture ...
10
votes
1answer
6k views

Different covariance types for Gaussian Mixture Models

While trying Gaussian Mixture Models here, I found these 4 types of covariances. ...
1
vote
1answer
467 views

Why use EM algorithm instead of just plain old ML for mixture model?

Let's say I have some [multivariate] data and want to fit a GMM to it. So I have $P_x=\sum_{i=1}^{n}\alpha_i{N(x;\theta_i)}$, where $x$ is an observation from the data, $\theta_i$ is the mean and ...
0
votes
0answers
30 views

Any trick to swap order of determinant and matrix inverse operation?

Been thinking through fitting a kind of Gaussian mixture model in more of a neural network style (kind of similar to RNADE or RMADE by Larochelle, without going into details) and see that this could ...
0
votes
1answer
27 views

How to match sample points to a probabilistic model?

This might be a trivial question I have a probabilistic model, say a Gaussian mixture model of known parameters, and with that I have a set of defined sample points. I would like to know how likely ...
0
votes
1answer
90 views

Is it a known phenomenon for the variance of a component (GMM) to increase without stopping?

I know it can happen for it to decrease dramatically as it overfits on a single datapoint. But I've never read about a component "taking everything over". See the following images (circles are stddevs)...
0
votes
1answer
75 views

How to prevent the creation of redundant mixtures while training a GMM?

I'm currently trying to train a GMM(UBM) with 1024 Gaussian mixtures for speaker verification. However, after training the GMM, it appears that some mixtures are useless/redundant. (little to no ...
0
votes
1answer
30 views

Very steep decrease in information criteria for mixture models with more components

I am analyzing data using mixture modeling. When I plot the information criteria (the BIC) for a series of models (with different model specifications and different number of mixture components), I ...
6
votes
1answer
455 views

Fitting Gaussian mixture models with dirac delta functions

I was told that using gradient methods for Gaussian mixture models may end up with Dirac delta function(s). I hadn't thought of this problem before, but when I verify this, it does seem to be a ...
2
votes
1answer
450 views

Gaussian Mixture Model - marginal likelihood

I am studying gaussian mixture models. The first step defines the following equation. They then proceed to marginalize $z_n$ out My question is, how did they arrive at that equation ? Where did the ...
2
votes
3answers
1k views

What's the difference between Multivariate Gaussian and Mixture of Gaussians?

What's the difference between Multivariate Gaussian and Mixture of Gaussians? If I have a Multivariate Gaussian and making all the data into ONE vector, is that a Mixture of Gaussians in 1 dimension?...
3
votes
1answer
723 views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
1
vote
0answers
266 views

Using Gaussian Mixture Model for outlier detection

Can someone summarize how to use Gaussian Mixture Model for outlier detection purpose. I am more interested in a general method and not so much the mathematical aspects of GMM's
1
vote
1answer
74 views

Estimating a GCD

I have two related questions. Question 1: Let $k_1, \ldots k_n$ be positive integers, and $\alpha_1, \ldots \alpha_n \in (0,1)$ be such that $\sum_{j \leq n} \alpha_j = 1$. Suppose $\langle X_m \...
0
votes
0answers
181 views

Mixture modelling with skewed distributions

I am trying to find a R library that splits a distribution into a symmetric and asymmetric components. I have a distribution that I want to split into two components, one skewed and the other most ...
1
vote
1answer
249 views

Gradient Ascent for Mixture of Gaussians

Beginner here, apologies if this is something very simple. I am trying to do a gradient ascent to estimate means for a Mixture of Gaussian model. I am using $(x-µ)/σ^3(2π)^{(1/2)} * e^{-((x-µ)/σ)^2)/2}...
0
votes
0answers
91 views

Mixture model with covariance matrix with varying variances and equal covariances

In some illustrations of mixture models, the covariance matrix is structured to have varying variances and equal covariances between mixture components. For example, Pastor et al. (2007) (I'm sorry ...
1
vote
2answers
145 views

PCA for probability vectors

Is there a procedure equivalent to principal component analysis (PCA) for probability vectors? I have an n-by-m array where every column sums to one, and all entries are positive. PCA works in ...
1
vote
0answers
57 views

How to interpret profiles / mixture components without any observations classified to them (when using MCLUST in R)

I am using the MCLUST software in R to fit normal mixture models, as part of what in my field are commonly called Latent Profile Analysis. Some of the time, ...
1
vote
1answer
44 views

Is there a way to accelerate expectation maximization?

There is a way to run a faster $k$-means by using Elkan's method, which uses the triangle inequality to avoid some calculations. I am trying to think of a way you could do a similar sort of thing for ...
2
votes
1answer
316 views

Use gradient methods for maximum likelihood estimation of Gaussian mixture

I have some questions concerning estimating maximum likelihood of Gaussian mixture model. As I have read around some material, they usually use EM algorithm for maximizing the complete likelihood ...
0
votes
1answer
58 views

Why the sample method of mixture distribution works?

For example this thread: Generating random variables from a mixture of Normal distributions First choose a distribution according to the weights. Then sample from the chosen distribution. How to ...
0
votes
0answers
437 views

Why did log-likelihood decrease in EM in this step-by-step example?

I'm using Expectation Maximization algorithm to determine the parameters of Gaussian distributions in a mixture. To get a better understanding of the algorithm, I executed it manually step by step on ...
6
votes
1answer
125 views

Observations for a bivariate Gaussian mixture

Consider two random vectors $X\equiv(X_1, X_2),Y\equiv(Y_1, Y_2)$ distributed as below 1) $X\sim N(\begin{pmatrix} \mu_{X,1}\\ \mu_{X,2}\\ \end{pmatrix}, \begin{pmatrix} v_{X,1} & 0\\ 0 & v_{...
4
votes
1answer
338 views

EM algorithm gaussian mixtures- derivation

I'm trying to make sense of a derivation I'm following from the lecture notes of Stanford's ML course. Specifically the notes are here: http://cs229.stanford.edu/notes/cs229-notes8.pdf I'm ...
0
votes
1answer
20 views

How to choose closer gaussian?

Suppose, I have two random number generators with normal distribution, which generated a number. How to decide, to which one it is most probably belong? Should I calculate probability density of each ...
1
vote
1answer
211 views

Generate data points for a Gaussian with drawing probability

I am trying to solve this question: Generate 500 data points drawn from each of 3 (three) Gaussians: $N_1(1, 0.1)$, $N_2(1.5., 0.1)$ and $N_3(2, 0.2)$ whose drawing probability on each iteration ...
0
votes
1answer
76 views

Does it make sense to have more mixtures than classes in a GMM

If I have M classes, does it ever make sense to have a Gaussian Mixture Model with K>M components, with multiple components predicting one class? I see this as having multiple Gaussians learning ...
1
vote
1answer
438 views

How to do batch learning for Gaussian Mixture Models?

I have a huge dataset of features on which I want to fit a Gaussian Mixture Model using standard expectation maximization, as it is implemented by sklearn. Since not all features fit into the memory ...
0
votes
1answer
206 views

Multi dimensionnal Gaussian Mixture Model

I'm a little confused about the GMM. I need to make clusterization. Actually, I have four dimensions : speed in Km/H, mileAge in Km, acceleration in m/s^2 and braking in m/s(^2). Each of those ...