Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
2
votes
1answer
99 views

Introduction to Gaussian mixture models

First of all, I am sorry if this question is not acceptable by some of the readers. However, I really read many, many sources about Gaussian mixture models, but all what I found was a short tutorial ...
2
votes
1answer
365 views

Gaussian Mixture Model

with the following code I fit a Gaussian Mixture Model to arbitrarily created data. The code is working. The only thing I encounter is that during the calculation of the multivariate_normal I ...
2
votes
2answers
218 views

This is Gaussian mixture model?

Here is a problem that I am looking at. Is this model really commonly known as a Gaussian mixture model (the one often appears as an illustration of EM algorithm)? I am confused because Gaussian ...
2
votes
1answer
478 views

Gaussian Mixture Model - marginal likelihood

I am studying gaussian mixture models. The first step defines the following equation. They then proceed to marginalize $z_n$ out My question is, how did they arrive at that equation ? Where did the ...
2
votes
1answer
349 views

Use gradient methods for maximum likelihood estimation of Gaussian mixture

I have some questions concerning estimating maximum likelihood of Gaussian mixture model. As I have read around some material, they usually use EM algorithm for maximizing the complete likelihood ...
2
votes
1answer
964 views

How to deal with categorical feature in a Gaussian Mixture model clustering model

I am performing clustering by Gaussian Mixture model using EM algorithm in R. U use the mclust package. My data (205 observations and 25 variables) has both ...
2
votes
1answer
113 views

Useful separation value in a mixture distribution

Assume we have a distribution that is the mixture of two normal distributions. The pdf of the overall distributions and their single parts may look like the following. In black, the combined ...
2
votes
1answer
500 views

Mixture Density Network: What is C?

I'm currently trying to implement a Mixture Density Network (MDN) based off of the original paper here. Most of the equations seem pretty straight forward but on page 6 (7 of the PDF) equation 23 has ...
2
votes
2answers
598 views

The pointwise product of densities of a Gaussian mixuture and a Gaussian

Let's say that I have a mixture of Gaussians representing a likelihood: $$ p(\mathbf{x}) = \sum_{i=1}^K\phi_i \mathcal{N}(\boldsymbol{\mu_i,\Sigma_i}) $$ What is the posterior distribution given a ...
2
votes
1answer
358 views

One-class Classification of multidimensional vectors

I have a m x k User-feature matrix (m >> k) obtained by factorizing an original User-websites matrix (m x n) that has #page views as entries. Additionally, there are users (say r) who have been ...
2
votes
2answers
3k views

Gaussian Naive Bayes really equivalent to GMM with diagonal covariance matrices?

Murphy writes that a multivariate Gaussian used in a generative classifier ("Gaussian discriminant analysis"), i.e., $p(\mathbf x|y=c,\mathbf\theta) = \mathcal{N}(\mathbf x|\mathbf y_c,\mathbf \...
2
votes
2answers
341 views

How to accomplish unsupervised separation of subpopulations?

I have a dataset drawn from a social network that looks Bimodal on logarithmic scales for all attributes (I'll demonstrate only one here): I know the variable that would give me a clean separation ...
2
votes
1answer
2k views

Number of free parameters in Gaussian mixture models

When comparing GMM models with different number of components (i.e number of Gaussians) one penalizes the likelihood for the total number of free parameters in the mixture model. If the data is in $D$ ...
2
votes
1answer
2k views

Implementing Gaussian mixture model for a HMM library

I'm working on an alignment algorithm using LAMP HMM library. This library supports Gaussian probability distribution but it does not seem to support Gaussian Mixture Model. What I want is, to input ...
2
votes
1answer
751 views

GMM with Bayes decision model

Given two classes of training data (A and B), I want to fit each class' distribution using a GMM with k components, and then use Bayes Decision Model for the classification. The first step was to ...
2
votes
0answers
35 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
2
votes
0answers
25 views

In a normal mixture model, what general happens if the number of groups to be summed far exceeds the dimensions of each normal?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_kI_d) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean vector $\mu$ and d-...
2
votes
0answers
35 views

Multimodality of mixtures of more than two Normal distributions

Let $$\phi(x;\mu,\sigma) = \frac{1}{\sigma \sqrt{2\pi}} \exp \left(- \frac{(x-\mu)^2}{2\sigma^2}\right)$$ denote the Gaussian density function ($\sigma > 0$). Let $$f(x) = \sum_{i=1}^N p_i \...
2
votes
1answer
346 views

GMM EM algorithm complexity per iteration

I was fitting GMM clusters with diagonal covariance on my data using EM with $n$ (=5e6) points, each having $m$ (=160) ...
2
votes
0answers
71 views

Gaussian Mixture Division

In the study of probabilistic graphical models (PGMs), the loopy belief update propagation (LBUP) message passing algorithm requires the division of unnormalised probability distributions. If the ...
2
votes
0answers
64 views

Distribution of the initializing set at K-means++

There is a well-known modification of the initializing step of K-means, named K-means++. It chooses cluster centers with probability proportional to its squared distance from the point's closest ...
2
votes
0answers
396 views

Approximating Uniform Distribution with Mixture of Gaussians

Let $T$ be a compact, connected, proper subset of $\mathbb{R}^3:\quad T \subset \mathbb{R}^3$. Further let $\left\{ \boldsymbol{\mu}_i \right\}_{i=1}^n$ be a given finite set of $n$ point in $T$: $$ \...
2
votes
0answers
540 views

Programming a mixture of a Gamma with a Normal distribution using R

I have some data x in R which seems to be a mixture of a Gamma and Normal distribution. Therefore I'd like to model this as a mixture model consisting of said distributions, but I don't know how to ...
2
votes
1answer
220 views

model selection, mixture of Gaussians

I have data and I want to decide whether it comes from 5-modal-normal distribution or 2-modal-normal distribution. In other words I want to check if it has 2 peaks or 5. I can estimate the $\mu$ and $\...
2
votes
1answer
222 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
2
votes
0answers
53 views

Variance of mix of normals

Suppose we have $n$ random variables distributed normally with the same mean and different variances. Suppose we know these variances. Which will be the variance of the marginal distribution induced ...
2
votes
0answers
416 views

Can a Gaussian mixture model be specified using a regression equation?

From: https://stats.stackexchange.com/a/236297/22199, I quote A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A ...
2
votes
0answers
150 views

Which cluster analysis for ordinal temporal data?

I would like to perform a cluster analysis but I’m not sure which is the best algorithm to apply to my data. My dataset is made of 200 cases (but the sample size can be enlarged). For each case, I ...
2
votes
0answers
873 views

Entropy of a set of categorical variables

In the context of Expectation-Maximization, I would like to compute te entropy factor in order to get the value of the lower bound when the algorithm converged. This lower bound can be expressed as: ...
2
votes
0answers
114 views

Expectation Maximization Gaussian Mixture Example

I am a biologist trying to understand expectation maximization for a mixture of two Gaussian distributions. I think I understand how to deal with the means of the two distributions, but I don't know ...
2
votes
0answers
211 views

Confusion with EM Algorithm for Gaussian Mixture?

I am trying to learn EM Algorithm for Gaussian Mixture. But not able to understand few stuffs. This is what I have understood. Consider GMM with k components. $$ p( \mathbf{x}| \mathbf{\alpha_{k}},\...
2
votes
0answers
163 views

Particle Filter and Gaussian Mixture

Let an observation model be given as $f(y_t|x_t)$ - this pdf is assumed to be nontrivial (not normal, not linear). The observation model is assumed to be known. Despite there is a state evolution ...
2
votes
0answers
268 views

Calculate number of standard deviations separating two multivariate Gaussians?

Given a set of multivariate Gaussian distributions (from fitting a Gaussian mixture model) I would like to be able to calculate the likelihood that a data point drawn from one Gaussian will improperly ...
2
votes
1answer
381 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian (...
2
votes
0answers
95 views

How can I measure separability between different number of instance of one feature vector?

How can I measure separability between different number of instance of one feature vector? For example the main vector is V=[1 1 2 3 4 5 7 8 10 100 1000 99 999 54] ...
2
votes
0answers
442 views

Should I add noise to my truth data before before training my classifier?

My task is to develop a system that will take in a series of measurements and return the probability that an object is a type 1, type 2,... type n. I will refer to the system I have to create as a ...
2
votes
1answer
812 views

Estimate Gaussian (mixture) density from a set of weighted samples

I am trying to find if there exists a way to find the spatial distribution over a set of points where the points are weighted. If I have "n" points in the (x,y) space, I can fit a mixture of Gaussians ...
1
vote
2answers
4k views

Standard deviation for weighted sum of normal distributions

I have 2 normally distributed random variable $H_0$ and $H_1$, which are combined to give the weighted distribution $H$ as follows: $H_0 \sim N(\mu_0, \sigma_0)$ $H_1 \sim N(\mu_1, \sigma_1)$ $$f_H ...
1
vote
1answer
272 views

Skewness of fitted mixture not correct?

I fitted a gaussian mixture to my financial data. The values are: $\pi= 0.3$ $\mu_1= -0.01$ $\mu_2= 0.01$ $\sigma_1=0.01$ $\sigma_2=0.03$ One can see, that both single distributions have a ...
1
vote
2answers
995 views

Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?

I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like: Since this was ...
1
vote
2answers
1k views

Mixed data in Gaussian Mixture Models

Is it possible to use a dataset with mixed variables such as continuous, ordered, and categorical variables and cluster the data using the Gaussian Mixed Model with EM algorithm. I cannot find ...
1
vote
1answer
69 views

How to use Bayes' Theorem to detect an event in a noisy signal

I'm trying to use Bayes' Theorem to solve a question that's come up in work, but I don't know if I've done it correctly, because the result seems a bit strange. The problem involves a stochastic ...
1
vote
1answer
3k views

The number of parameters in Gaussian mixture model

I have D-dimensional data with K components. How many parameters if I use a model with full covariance matrices? and How many if I use diaogonal covariance matrices?
1
vote
1answer
509 views

How to do batch learning for Gaussian Mixture Models?

I have a huge dataset of features on which I want to fit a Gaussian Mixture Model using standard expectation maximization, as it is implemented by sklearn. Since not all features fit into the memory ...
1
vote
1answer
292 views

Bimodal univariate distributions are always indicative of a mixture of two random variables. Is this correct? [duplicate]

Say I see a bimodal distribution like this (with the domain, or random variable, $Z$): Does that instantly mean that I am seeing not a distribution of one independent random variable $Z$, but ...
1
vote
1answer
94 views

What does 'vector-valued' mean?

What is the difference of a feature vector and a 'vector-valued observation' as described here? The term 'vector-valued' is used in the following context: "Most state-of-the-art [Automatic Speech ...
1
vote
1answer
11k views

Generate sample data from Gaussian mixture model [duplicate]

I am given the values for mean, co-variance, initial_weights for a mixture of Gaussian Models. Now how can I generate samples given those: In brief, I need a function like ...
1
vote
1answer
151 views

Mixture Model Distributions

I wonder, if there could be a Pareto Mixture Model, just like the Gaussian Mixture Model (GMM). How am I supposed to build a Pareto Mixture Model (PMM)?
1
vote
1answer
114 views

I want to show a local optimum in my paper, how do I generate the data for it?

I'm writing a paper where I am explaining the problems of local optimum in my clustering algorithm. While clustering, in my data I would at times get local optimums. But I've tried and I cannot ...
1
vote
1answer
539 views

My MCMC do not overlap : Mixturemodel with JAGS and R

I fitted a JAGS model and I have those results : My questions are: Why do my chains not overlap, and how can I fix that? I used the following method: My model is a mixture Gaussian model of two ...