Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
2
votes
1answer
161 views

Gaussian Mixture and K-Means ?! a big challenge?

This is taken from Tom. Mitche Material as Old-Exam. I think the (2) is true and not (3). Who can verify me?
2
votes
2answers
89 views

Re-estimating a probability distribution with additional priors

I have a 3D dataset with at least millions of data points (scatter events from atoms, approximately Gaussian). I am modeling this data with a Gaussian Mixture Model. The usual approach would be to ...
2
votes
1answer
2k views

matlab gmdistribution.fit 'Regularize' - what regularization method?

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
2
votes
1answer
96 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
2
votes
1answer
28 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
2
votes
1answer
43 views

Gibbs sampling allocations for time dependent observations from this model

I observe $N$ observations $\{x_{1,t_1}, \dots, x_{N,t_N}\}$ from a $k$ component Gaussian Mixture model. The $i$th observation is seen at time stamp $t_i$ and is distributed such that each $x_{i,t_i}|...
2
votes
1answer
77 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
53 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
2
votes
1answer
206 views

Correlated random variables from mixture distributions

Let I have three random variables whose density is a mixture of two Normals with these parameters: $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$ $\...
2
votes
1answer
121 views

Introduction to Gaussian mixture models

First of all, I am sorry if this question is not acceptable by some of the readers. However, I really read many, many sources about Gaussian mixture models, but all what I found was a short tutorial ...
2
votes
1answer
464 views

Gaussian Mixture Model

with the following code I fit a Gaussian Mixture Model to arbitrarily created data. The code is working. The only thing I encounter is that during the calculation of the multivariate_normal I ...
2
votes
2answers
230 views

This is Gaussian mixture model?

Here is a problem that I am looking at. Is this model really commonly known as a Gaussian mixture model (the one often appears as an illustration of EM algorithm)? I am confused because Gaussian ...
2
votes
1answer
543 views

Gaussian Mixture Model - marginal likelihood

I am studying gaussian mixture models. The first step defines the following equation. They then proceed to marginalize $z_n$ out My question is, how did they arrive at that equation ? Where did the ...
2
votes
1answer
454 views

Use gradient methods for maximum likelihood estimation of Gaussian mixture

I have some questions concerning estimating maximum likelihood of Gaussian mixture model. As I have read around some material, they usually use EM algorithm for maximizing the complete likelihood ...
2
votes
1answer
145 views

Useful separation value in a mixture distribution

Assume we have a distribution that is the mixture of two normal distributions. The pdf of the overall distributions and their single parts may look like the following. In black, the combined ...
2
votes
1answer
507 views

Mixture Density Network: What is C?

I'm currently trying to implement a Mixture Density Network (MDN) based off of the original paper here. Most of the equations seem pretty straight forward but on page 6 (7 of the PDF) equation 23 has ...
2
votes
2answers
619 views

The pointwise product of densities of a Gaussian mixuture and a Gaussian

Let's say that I have a mixture of Gaussians representing a likelihood: $$ p(\mathbf{x}) = \sum_{i=1}^K\phi_i \mathcal{N}(\boldsymbol{\mu_i,\Sigma_i}) $$ What is the posterior distribution given a ...
2
votes
1answer
392 views

One-class Classification of multidimensional vectors

I have a m x k User-feature matrix (m >> k) obtained by factorizing an original User-websites matrix (m x n) that has #page views as entries. Additionally, there are users (say r) who have been ...
2
votes
2answers
3k views

Gaussian Naive Bayes really equivalent to GMM with diagonal covariance matrices?

Murphy writes that a multivariate Gaussian used in a generative classifier ("Gaussian discriminant analysis"), i.e., $p(\mathbf x|y=c,\mathbf\theta) = \mathcal{N}(\mathbf x|\mathbf y_c,\mathbf \...
2
votes
2answers
352 views

How to accomplish unsupervised separation of subpopulations?

I have a dataset drawn from a social network that looks Bimodal on logarithmic scales for all attributes (I'll demonstrate only one here): I know the variable that would give me a clean separation ...
2
votes
1answer
2k views

Number of free parameters in Gaussian mixture models

When comparing GMM models with different number of components (i.e number of Gaussians) one penalizes the likelihood for the total number of free parameters in the mixture model. If the data is in $D$ ...
2
votes
1answer
2k views

Implementing Gaussian mixture model for a HMM library

I'm working on an alignment algorithm using LAMP HMM library. This library supports Gaussian probability distribution but it does not seem to support Gaussian Mixture Model. What I want is, to input ...
2
votes
1answer
756 views

GMM with Bayes decision model

Given two classes of training data (A and B), I want to fit each class' distribution using a GMM with k components, and then use Bayes Decision Model for the classification. The first step was to ...
2
votes
1answer
98 views

Conditional Probability - Mixture Model

I know that the likelihood in a p-dimensional Gaussian mixture model is given by $$ p(s|\theta) = \sum_{b_1 = 0}^1\cdots\sum_{b_p = 0}^1\left[ \prod_{i=1}^pw^{1-b_i}(1-w)^{1-b_i}\right]\phi_p(s|\mu(b,...
2
votes
0answers
37 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
2
votes
0answers
81 views

Statistical test for comparing two-gaussian mixture

I have a distribution of shape sizes under two different (biological) conditions. From prior knowledge, I do expect there to be two populations. I fit each condition to a two-Gaussian mixture model. ...
2
votes
0answers
25 views

In a normal mixture model, what general happens if the number of groups to be summed far exceeds the dimensions of each normal?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_kI_d) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean vector $\mu$ and d-...
2
votes
0answers
35 views

Multimodality of mixtures of more than two Normal distributions

Let $$\phi(x;\mu,\sigma) = \frac{1}{\sigma \sqrt{2\pi}} \exp \left(- \frac{(x-\mu)^2}{2\sigma^2}\right)$$ denote the Gaussian density function ($\sigma > 0$). Let $$f(x) = \sum_{i=1}^N p_i \...
2
votes
1answer
422 views

GMM EM algorithm complexity per iteration

I was fitting GMM clusters with diagonal covariance on my data using EM with $n$ (=5e6) points, each having $m$ (=160) ...
2
votes
0answers
73 views

Gaussian Mixture Division

In the study of probabilistic graphical models (PGMs), the loopy belief update propagation (LBUP) message passing algorithm requires the division of unnormalised probability distributions. If the ...
2
votes
2answers
169 views

PCA for probability vectors

Is there a procedure equivalent to principal component analysis (PCA) for probability vectors? I have an n-by-m array where every column sums to one, and all entries are positive. PCA works in ...
2
votes
0answers
66 views

Distribution of the initializing set at K-means++

There is a well-known modification of the initializing step of K-means, named K-means++. It chooses cluster centers with probability proportional to its squared distance from the point's closest ...
2
votes
0answers
422 views

Approximating Uniform Distribution with Mixture of Gaussians

Let $T$ be a compact, connected, proper subset of $\mathbb{R}^3:\quad T \subset \mathbb{R}^3$. Further let $\left\{ \boldsymbol{\mu}_i \right\}_{i=1}^n$ be a given finite set of $n$ point in $T$: $$ \...
2
votes
0answers
578 views

Programming a mixture of a Gamma with a Normal distribution using R

I have some data x in R which seems to be a mixture of a Gamma and Normal distribution. Therefore I'd like to model this as a mixture model consisting of said distributions, but I don't know how to ...
2
votes
1answer
232 views

model selection, mixture of Gaussians

I have data and I want to decide whether it comes from 5-modal-normal distribution or 2-modal-normal distribution. In other words I want to check if it has 2 peaks or 5. I can estimate the $\mu$ and $\...
2
votes
1answer
238 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
2
votes
0answers
56 views

Variance of mix of normals

Suppose we have $n$ random variables distributed normally with the same mean and different variances. Suppose we know these variances. Which will be the variance of the marginal distribution induced ...
2
votes
0answers
470 views

Can a Gaussian mixture model be specified using a regression equation?

From: https://stats.stackexchange.com/a/236297/22199, I quote A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A ...
2
votes
0answers
156 views

Which cluster analysis for ordinal temporal data?

I would like to perform a cluster analysis but I’m not sure which is the best algorithm to apply to my data. My dataset is made of 200 cases (but the sample size can be enlarged). For each case, I ...
2
votes
0answers
996 views

Entropy of a set of categorical variables

In the context of Expectation-Maximization, I would like to compute te entropy factor in order to get the value of the lower bound when the algorithm converged. This lower bound can be expressed as: ...
2
votes
0answers
115 views

Expectation Maximization Gaussian Mixture Example

I am a biologist trying to understand expectation maximization for a mixture of two Gaussian distributions. I think I understand how to deal with the means of the two distributions, but I don't know ...
2
votes
0answers
130 views

Posterior pointwise uncertainty of multivariate normal-Wishart (variational GMM)

Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\...
2
votes
0answers
219 views

Confusion with EM Algorithm for Gaussian Mixture?

I am trying to learn EM Algorithm for Gaussian Mixture. But not able to understand few stuffs. This is what I have understood. Consider GMM with k components. $$ p( \mathbf{x}| \mathbf{\alpha_{k}},\...
2
votes
0answers
164 views

Particle Filter and Gaussian Mixture

Let an observation model be given as $f(y_t|x_t)$ - this pdf is assumed to be nontrivial (not normal, not linear). The observation model is assumed to be known. Despite there is a state evolution ...
2
votes
0answers
272 views

Calculate number of standard deviations separating two multivariate Gaussians?

Given a set of multivariate Gaussian distributions (from fitting a Gaussian mixture model) I would like to be able to calculate the likelihood that a data point drawn from one Gaussian will improperly ...
2
votes
1answer
418 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian (...
2
votes
0answers
95 views

How can I measure separability between different number of instance of one feature vector?

How can I measure separability between different number of instance of one feature vector? For example the main vector is V=[1 1 2 3 4 5 7 8 10 100 1000 99 999 54] ...
2
votes
0answers
444 views

Should I add noise to my truth data before before training my classifier?

My task is to develop a system that will take in a series of measurements and return the probability that an object is a type 1, type 2,... type n. I will refer to the system I have to create as a ...
2
votes
1answer
863 views

Estimate Gaussian (mixture) density from a set of weighted samples

I am trying to find if there exists a way to find the spatial distribution over a set of points where the points are weighted. If I have "n" points in the (x,y) space, I can fit a mixture of Gaussians ...
1
vote
1answer
12k views

Generate sample data from Gaussian mixture model [duplicate]

I am given the values for mean, co-variance, initial_weights for a mixture of Gaussian Models. Now how can I generate samples given those: In brief, I need a function like ...