Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
3
votes
0answers
1k views

Fit Gaussian Mixture model directly to the mixture density

The core of the question is: Can I estimate the parameters of a gaussian mixture model (with EM or Dirichlet Process) from a mixture density directly, that is, without using data drawn from such ...
3
votes
0answers
546 views

How to calculate BIC for multidimensional problem

Sorry for this question, but I am really not sure how to calculate BIC for my situation. My models are mixtures of normals with different number of components. Variances are equal for all components ...
3
votes
0answers
165 views

Posterior pointwise uncertainty of multivariate normal-Wishart (variational GMM)

Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\...
3
votes
1answer
222 views

What is the link (discrepancy?) between these PDF/CDF and p-value distributions?

I have created a mixed distribution model comprising 80% $H_0$ plus 20% $H_1$ to illustrate the link between the expected proportions of true and false positives and negatives in the PDF, CDF and p-...
3
votes
0answers
498 views

Diffusion coefficient from double-normal probability density function

The spread of individuals of species is often described by so-called dispersal kernels. The main parameter of spread is then often the variance defined as the average squared movement distance of a ...
3
votes
0answers
326 views

Why isn't a gaussian mixture prone to overfitting?

Consider a Gaussian mixture of 2 components and a dataset of size $N$. The EM algorithm use the data to estimate: the model parameters: the means $\mu_1, \mu_2$ (say the covariances matrices are ...
3
votes
0answers
436 views

A better bayesian way of modelling autoregressive mixtures

I have a JAGS hierarchical model which includes a temporal sub-model for the primary vote share between four party groups (LNP, Labor, Green, and Other). For each day in the temporal model, the vote ...
2
votes
2answers
358 views

AICc is picking overly complex models - something stricter?

I'd like to know if there are stricter alternatives to automated model selection than AICc / AIC / BIC. We have approximately ten thousand curves, and for each we'd like to find the most parsimonious ...
2
votes
1answer
2k views

A Gaussian Mixture Model Is a Universal Approximator of Densities

When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following: A Gaussian mixture model is a universal approximator of densities, in the ...
2
votes
2answers
271 views

interpretation of the estimated parameters of a gaussian mixture model

I need to find/fit a model for the color of an object. Suppose its color is generally yellow and we have 10000-by-3 data which are pixel values for R, G, B channels. Firstly I choose a Multivariate ...
2
votes
1answer
20k views

Generate sample data from Gaussian mixture model [duplicate]

I am given the values for mean, co-variance, initial_weights for a mixture of Gaussian Models. Now how can I generate samples given those: In brief, I need a function like ...
2
votes
1answer
612 views

Sampling from Gaussian mixture models, when are the sampled data independent?

Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions $p(x) = \sum\limits_{i = 1}^N w_i \mathcal{N}(x;\mu_i, \Sigma_i)$ where $w_i$ are the weights. Now I sample some points $\...
2
votes
1answer
2k views

Gaussian Mixture: is this plot right?

I'm studying about Gaussian Mixtures and I decided to play around with it in Python, but I'm not entirely sure if I understand it fully. I generated some data, and then calculated the Gaussian ...
2
votes
2answers
4k views

What is the best function to fit onto a “flat top gaussian”?

I am studying some signal and I am trying to make an automated algorithm that can extract the parameters for my signal. First let me describe a bit, I have a light emiting object crossing a window of ...
2
votes
2answers
210 views

fitting a Gaussian mixture with a constraint in python

Suppose I have data and I want to fit a two component Gaussian mixture to it. I don't know how to do it in python but worse than that is that I have an additional constraint that the mean of one ...
2
votes
2answers
1k views

Compute quantile function from a mixture of Normal distribution

I have this mixture of normal distribution: $$X \sim \frac{1}{2}\mathcal{N}(\mu_{x_1}=10,\,\sigma_{x_1}^{2}=1)+\frac{1}{2}\mathcal{N}(\mu_{x_2}=13,\,\sigma_{x_2}^{2}=1)$$ How can i compute the ...
2
votes
1answer
764 views

Conditional distribution for Gibbs sampling for Gaussian mixture

If we draw $n$ i.i.d. points $x_1,x_2,\dots,x_n$ from the following Gaussian mixture: $$ \frac 12 \mathcal N(x \mid \mu_1,1) + \frac 12 \mathcal N(x\mid \mu_2,1) $$ and the prior $p(\mu_1 , \mu_2 )$ ...
2
votes
1answer
469 views

Doesn't the non-Gaussian source assumption of ICA render it practically useless?

Gaussian distributions appear everywhere in nature, indeed this was largely the justification for most classical methods' reliance on assumption of normality. ICA assumes non-Gaussian sources, indeed ...
2
votes
1answer
2k views

When would I use EM instead of k-means?

When would I want to assign cluster probabilities to patterns instead of hard assignments to clusters? Can someone elaborate?
2
votes
2answers
123 views

Interpreting mixture of Gaussians (Variational Inference)

I've recently stated reading about mixture models and variational inference in this excellent paper, but I'm having troubles dissecting the models described, and have a couple of questions. Please see ...
2
votes
1answer
7k views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
2
votes
1answer
416 views

what is marginal density based on this picture?

I am studying Gaussian mixtures from the book Pattern Recognition and Machine Learning by Chris Bishop. Figure (a) are contours of the mixture components (Gaussians) with the corresponding weights. ...
2
votes
2answers
650 views

Why are hidden Markov models considered 'mixture models'?

I'm a bit confused by the conception of "mixture model" I'm studying hidden Markov model, which is frequently referred to as a "mixture model". But I don't know what the term "mixture" implies. ...
2
votes
1answer
110 views

Scikit learn - GMM log-likelihood. Why use Cholesky's precision matrix instead of covariance matrix?

This is my first post, please let me know if I am not being clear. I am trying to understand the sklearn.mixture.GaussianMixture.score(X). As I understand that the ...
2
votes
1answer
179 views

Figueiredo and Jain's Gaussian mixture EM convergence criterion

I have implemented and been playing around Figueiredo & Jain 's trainer in this paper http://www.lx.it.pt/~mtf/IEEE_TPAMI_2002.pdf for Gaussian mixture. Fig. 2 in the paper depicts the full ...
2
votes
1answer
183 views

Gaussian mixture distribution confusion

Have some confusion on Gaussian mixture distribution model (reference is from the book of Pattern Recognition and Machine Learning). My confusion is how below formula works? $\sum_kz_k=1$ My thought ...
2
votes
1answer
106 views

What does 'vector-valued' mean?

What is the difference of a feature vector and a 'vector-valued observation' as described here? The term 'vector-valued' is used in the following context: "Most state-of-the-art [Automatic Speech ...
2
votes
2answers
337 views

Applying Akaike Information Criterion on collection of Gaussian fits

I am trying to apply Akaike Information Criterion on a collection of Gaussian mixtures fitted on some data points. My question is, can I use AIC even if the number of components of Gaussian mixtures ...
2
votes
1answer
2k views

Time varying Gaussian distribution

Does there exist time varying Gaussian model? To be specific, a 2D Gaussian model whose mean and covariance matrix is varying with another parameter called time. The $\mu$(mean) and $\sigma$(...
2
votes
1answer
312 views

Best approach to classifying 3-point trajectories?

I have a sample of about 300 subjects who have been measured at 3 different times (morning, afternoon, evening). The variable of interest can be assumed to be approximately normal. It appears that ...
2
votes
2answers
285 views

Whitening a mixture of Gaussians

Suppose I have a dataset that was sampled from a mixture of Gaussians: $$ X \sim \sum_i w_i \mathcal{N}(\mu_i, \Sigma_i)$$ Technically, I can center and then whiten $X$ so that it has zero mean and ...
2
votes
2answers
145 views

EM-algorithm for two clusters (when one of the distributions is uniform)

I am having a hard time with the EM-algorithm. Here's the problem that I am trying to solve. Dealing with noisy annotations is a common problem in computer vision, especially when using ...
2
votes
1answer
348 views

Conditional Probability - Mixture Model

I know that the likelihood in a p-dimensional Gaussian mixture model is given by $$ p(s|\theta) = \sum_{b_1 = 0}^1\cdots\sum_{b_p = 0}^1\left[ \prod_{i=1}^pw^{1-b_i}(1-w)^{1-b_i}\right]\phi_p(s|\mu(b,...
2
votes
1answer
395 views

Distribution of Distributions | percent point function

I have the following normal distribution (Primary Distribution): Mean = 7 Sigma = 0.5 and a list of other normal distributions (Secondary Distributions): (python list of tupels, each tupel ...
2
votes
2answers
113 views

Can Gaussian Mixture Model Clustering tell me something about the distribution of my data?

I have 10,000 vectors originating from 5 separate classes (2,000 each). I use Gaussian Mixture Model clustering (in Python) to cluster the 10,000 vectors, telling the algorithm to cluster the data ...
2
votes
1answer
379 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
2
votes
1answer
2k views

Assessing Gaussian mixture distribution by cross validation

I have a 10 dimensional random vector that I'm modelling with GMMs. I want to estimate the best number of mixtures ($K$) for my data via the following method: Divide the data to train (90%) and test (...
2
votes
1answer
377 views

Need intuition - how do they simplify the Q function for gaussian mixture EM?

Background - I'm trying to follow section 7.2.4 in this EM tutorial. Basically the setup is I have a vector with 10 points $x$, and each of them can be assigned ($y$) to either Gaussian 1 or to ...
2
votes
1answer
1k views

How to know if my Gaussian mixture model has enough training data?

A somewhat soft question - I'm training a Gaussian mixture model (with the EM algorithm) on data of size $N$ ($N$ is typically between 4 and 64). How much samples do I need? obviously it depends on ...
2
votes
1answer
2k views

R clustering using mclust: BIC are often NA

I'm working on segmentation/clustering and trying to use Gaussian Mixture Modelling for Model-Based Clustering. I'm using the R package Mclust in order to come up with the best fit for my data. All ...
2
votes
1answer
537 views

Bayesian Gaussian Mixture model

I am trying to fit basic Gaussian mixture with a Bayesian model. My likelihood function is Gaussian, with std=1, and the only parameter is the mean, chosen from $\{0,1,\dots,14,15\}$ and my prior is ...
2
votes
2answers
307 views

Mixture Model with dependant observations

I am trying to model a process in which each datapoint is generated sequentially, so the current observation depends on the last one. Some example data could look like, ...
2
votes
2answers
94 views

Re-estimating a probability distribution with additional priors

I have a 3D dataset with at least millions of data points (scatter events from atoms, approximately Gaussian). I am modeling this data with a Gaussian Mixture Model. The usual approach would be to ...
2
votes
2answers
5k views

Gaussian Naive Bayes really equivalent to GMM with diagonal covariance matrices?

Murphy writes that a multivariate Gaussian used in a generative classifier ("Gaussian discriminant analysis"), i.e., $p(\mathbf x|y=c,\mathbf\theta) = \mathcal{N}(\mathbf x|\mathbf y_c,\mathbf \...
2
votes
2answers
508 views

Maximum likelihood estimation get biased by fat-tail samples identifying mixture model of 2 normal distributions

I'm building a mixture model consists 2 normal distribution: \begin{equation} X_1 \sim \mathcal{N}(\mu_1,\sigma_{1}^{2}), \quad X_2 \sim \mathcal{N}(\mu_2,\sigma_{2}^{2})\end{equation} while $$pdf(...
2
votes
1answer
43 views

Is this equivalent to a Gaussian mixture model?

Similar to how you can derive the normal distribution as the distribution where the probability near a point exponentially decays with the square of the number of standard deviations you are away from ...
2
votes
2answers
68 views

Estimating weights of known component distributions in a mixture distribution

Given $n$ probability density functions ($p_1$, ..., $p_n$) with known distributions, what are the ways of estimating the weights ($w_1$, ..., $w_n$) of these component distributions given a sample ...
2
votes
1answer
152 views

Clarifying Dirichlet Process Mixture Probability Terms

Suppose I have a Dirichlet Process Mixture model defined as follows: $\alpha \sim G(a,b)\\ \pi|\alpha \sim \text{Dir}(\alpha)\\ z|\pi \sim \text{Cat}(\pi)\\ $ where $G$ is just a standard Gamma ...
2
votes
1answer
1k views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
2
votes
1answer
62 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...

1 2 3
4
5
12