Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

3
votes
1answer
218 views

Universal Approximation Capabilities of Mixture Models

I am looking for two reference incl. proofs showing 1) that a discrete Mixture of Gaussians can asymptotically approximate any (well behaved) continuous density 2) that a discrete Mixture of ...
2
votes
1answer
215 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
2
votes
1answer
781 views

Estimate Gaussian (mixture) density from a set of weighted samples

I am trying to find if there exists a way to find the spatial distribution over a set of points where the points are weighted. If I have "n" points in the (x,y) space, I can fit a mixture of Gaussians ...
2
votes
1answer
360 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian (...
1
vote
1answer
110 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
1
vote
1answer
71 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
1
vote
1answer
410 views

Anomaly detection on 1D data with multiple gaussian distributions

My core problem is to set a cutoff to my one dimension data between normal with abnormal. I think this is a 'anomaly detection' problem. My Data My data is one dimension, consists with below: (...
1
vote
1answer
74 views

Estimating a GCD

I have two related questions. Question 1: Let $k_1, \ldots k_n$ be positive integers, and $\alpha_1, \ldots \alpha_n \in (0,1)$ be such that $\sum_{j \leq n} \alpha_j = 1$. Suppose $\langle X_m \...
1
vote
1answer
44 views

Is there a way to accelerate expectation maximization?

There is a way to run a faster $k$-means by using Elkan's method, which uses the triangle inequality to avoid some calculations. I am trying to think of a way you could do a similar sort of thing for ...
0
votes
1answer
17 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
0
votes
1answer
129 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
0
votes
0answers
26 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
0
votes
0answers
24 views

GMM model of the joint distribution from multivariate marginals

I have two multivariate Gaussian variables $X \sim \mathcal{N}(\boldsymbol {\mu}_X \in \mathcal R^d,\boldsymbol {\Sigma}_X \in \mathcal R^{d \times d})$ and $Y \sim \mathcal{N}(\boldsymbol {\mu}_Y \...
0
votes
0answers
21 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
0
votes
0answers
5 views

Guassian Mixture Classification: interpretating component x variable means matrix

The Guassian Mixture model output by mclust::Mclust() function has a $parameters$mean element which is a matrix with dimensions ...
0
votes
0answers
77 views

Output Size of Mixture Density Networks

I am working on a neural network in which the final output layer will be a Mixture Density Network (MDN), but am confused about the shape of the values that final layer should return. In the paper in ...
0
votes
0answers
103 views

How to interpret a GMM and a predict_GMM output

I would like to do outlier detection in a dataset by using e Mixture Model. I'm using the R functions GMM and predict_GMM but I can't understand which is the outlier factor and how to indentify ...
0
votes
0answers
426 views

Calculating BIC for sklearn.mixture BayesianGaussianMixture

The GaussianMixture function easily allows calculation of BIC. In the source code it is defined as: ...
0
votes
0answers
76 views

How to select the best Baysian GMM with Dirichlet process

I am very new to GMM models and currently trying to make a unsupervised clustering on data. The data has 34 features (dimension). I am thinking of using GMM with Dirichlet process and trying to code ...
0
votes
0answers
55 views

Reducing the number of Gaussians in a Gaussian Mixture Model

I build a kernel density estimation (KDE) of Gaussian kernels. I have many samples, but the distribution is not too complicated. I think it should be possible to approximate the resulting KDE by a ...
0
votes
0answers
75 views

how to model hourly wind speed data

I am trying to forecast hourly wind speed (HWS) data in Trinidad and Tobago and I have read in the literature that "Direct application of stochastic models (ARMA & ARIMA models) to HWS series is ...
0
votes
0answers
30 views

Any trick to swap order of determinant and matrix inverse operation?

Been thinking through fitting a kind of Gaussian mixture model in more of a neural network style (kind of similar to RNADE or RMADE by Larochelle, without going into details) and see that this could ...
0
votes
0answers
181 views

Mixture modelling with skewed distributions

I am trying to find a R library that splits a distribution into a symmetric and asymmetric components. I have a distribution that I want to split into two components, one skewed and the other most ...
0
votes
0answers
91 views

Mixture model with covariance matrix with varying variances and equal covariances

In some illustrations of mixture models, the covariance matrix is structured to have varying variances and equal covariances between mixture components. For example, Pastor et al. (2007) (I'm sorry ...
0
votes
0answers
437 views

Why did log-likelihood decrease in EM in this step-by-step example?

I'm using Expectation Maximization algorithm to determine the parameters of Gaussian distributions in a mixture. To get a better understanding of the algorithm, I executed it manually step by step on ...
0
votes
0answers
100 views

Two variables in Multivariate Gaussian pdf

I have problem with computing multivariate gaussian probability density function(pdf) value. As I found the equation is, $$p(\textbf{x}|\mu, \Sigma) = \frac{1}{(2\pi )^{\frac{n}{2}} |\Sigma|^{\frac{1}...
0
votes
0answers
127 views

upper bound on number of components of GMM

I want to fit GMM to a data set with N data points in D dimensions. I am using the full GMM in MATLAB, i.e., each component has a complete covariance matrix (as opposed to diagonal covaraices which ...
0
votes
0answers
35 views

System of Gaussian equations

Let $N(x\ |\ \mu,\sigma^2)$ be the pdf of a normal random variable with mean $\mu$ and variance $\sigma^2$. Question: Given $n$ data points $(x_i,y_i)_{i=1,\dots,n}$, compute $\{w_i\}_i$,$\{\sigma_i^2\...
0
votes
0answers
257 views

Why is gradient used in Fisher Vectors?

My understanding of Fisher vectors can be described in the following manner: A GMM is trained on all data, which gives $p(X,\theta)$, then, for each image/video ($X_i$), the gradient of $p(X,\theta)$ ...
0
votes
0answers
134 views

Mixture Proposal Distributions

I have a target distribution $\mu$ which I would like to investigate using, for instance Metropolis-Hastings-Green (MHG). So, given a Gaussian prior, $\pi$, and a likelihood $L$ such that $\mu(dx) \...
0
votes
0answers
63 views

Given a pdf which is a mixture of Gaussians, how do I infer the position (mean), variance, and number of Gaussians?

I have the following data, which when plotted as a histogram, are a mixture of Gaussians: I would like to write an algorithm that would infer: (1) the number of "peaks" or normal distributions in ...
0
votes
0answers
628 views

Predicting intensity of Poisson process, given event data

I have a dataset of events: each row is an event, and each column is a feature. There are millions of events and several dozen features. The features are mostly numerical (a few are categorical and I ...
0
votes
0answers
284 views

How to explain the Gath-Geva algorithm for Gaussian mixture models intuitively?

I have used the Gath-Geva algorithm for my geophysical research. One of the reviewers responded that he does not understand the algorithm and asked for a flow chart. What should I do? I have explained ...
0
votes
0answers
209 views

mixture of Gaussians vs mixture of quadratic denominators (Cauchy)

It is known that mixture of Gaussians are dense in the set of all distribution functions. A 1-dimensional Gaussian has the following density: $$ \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(\omega-\beta)^...
0
votes
0answers
117 views

How to calculate the value for a multivariate Gaussian

How to evaluate the value for a multivariate Gaussian. For instance, to evaluate the 20 dimensional Gaussian function value with respect to a 20 dimensional input vector x, I need calculate a 20 by 20 ...
0
votes
0answers
250 views

Is it possible to convert a Gaussian Mixture Model implementation into a Categorical Mixture Model?

I am modelling whether a customer will spend when given a voucher. I have a theory that a customer falls into one of two latent classes: call them spendthrift and miser. So I would like to fit a ...
0
votes
0answers
93 views

Chi-square approximation in homogeneity

I am interested in testing homogeneity in mixture of Gaussians (testing no mixture vs. 2 populations) given that we know the weights of the two distributions. We can first use MLE to estimate the mean ...
0
votes
0answers
398 views

Gaussian Mixture Model with Custom Distance Metric

I have some 1D data that I want to cluster using Mixture of Gaussian. However, the data "wraps around" at two extremes. Specifically, I have a list of angles from $-\pi$ to $\pi$ and the data near two ...
0
votes
0answers
1k views

How to pool more than two sample means and standard deviations?

I have 4 independent samples from one population, with their respective sample sizes, means and standard deviations. I don't have the raw data, and all of them follow gaussian distributions. Sample 1:...
0
votes
0answers
716 views

Novelty and Outlier Detection in Unsupervised Learning Style

Currently I am looking for some method to do novelty and outlier detection. I found some good example here using scikit-learn (Link1). However, it is based on supervised learning and I believe the ...
0
votes
0answers
125 views

To model an unknown bounded probability density function by a Gaussian mixture

I have points in dimension 10 coming from an unknown probability distribution. The nature of data strongly suggests that this distribution is bounded. But the boundaries are not precisely known and ...
0
votes
0answers
1k views

Fitting for a Poisson-Gaussian Mixture Distribution

First of all, I am rather new to statistics, so go easy on me. I am aware that the negative binomial distribution can be thought to arise as a result of letting the $\lambda$ parameter in a Poisson ...
0
votes
0answers
38 views

Test is data is derived from mixture or just noisy normal

Is there a way to quantify \ test \ describe how likely it is that high dimensional data come from a single Gaussian or a mixture of Gaussians (with different means \ SDs) or not? What I am thinking ...
0
votes
0answers
169 views

Too good results from EM for gaussian mixture

My task is to identify parameters (mean, standard deviation, height) of gaussian peaks in given histogram data with as lowest CV as possible. Number of peaks and approximate means are known (pointed ...
0
votes
0answers
371 views

How to make the most of a Gaussian mixture assumption in a model?

I have a dataset with 100 columns and approximately 100000 lines. I have a variable to predict that is Y (0,1 so it's a classification problem). I have an other categorical variable with two values ...
0
votes
0answers
236 views

Approximating a binomial distribution with a mixture normal

This is purely a theoretical question (I legitimately can't think of a real application), but if you wanted to approximate a binomial distributed variable with a two-component mixture normal, is there ...
0
votes
0answers
2k views

How to use Gaussian mixture model for multivariate pattern classification

I am new to statistical pattern recognition and trying to learn.To begin with I am trying to work with two class problems and trying to classify motion activities as mentioned in the paper "Object ...
0
votes
0answers
634 views

How to decide whether the distribution is unimodal or bimodal in grain size distribution?

In the examples given at this link, I am not able to decide whether the distribution is unimodal or bimodal. I think it is in between unimodal and bimodal, but I do not know if this kind of class ...
-1
votes
0answers
31 views

Gaussian mixture model (GMM) to cluster high dimensional dataset

My data set has ~20.000 dimensions. I want to use "Gaussian mixture model " to cluster them. To Construct mixtures, multivariate distributions are required. Given that high dimensionality, how ...