Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
1
vote
1answer
274 views

Skewness of fitted mixture not correct?

I fitted a gaussian mixture to my financial data. The values are: $\pi= 0.3$ $\mu_1= -0.01$ $\mu_2= 0.01$ $\sigma_1=0.01$ $\sigma_2=0.03$ One can see, that both single distributions have a ...
1
vote
2answers
1k views

Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?

I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like: Since this was ...
1
vote
2answers
1k views

Mixed data in Gaussian Mixture Models

Is it possible to use a dataset with mixed variables such as continuous, ordered, and categorical variables and cluster the data using the Gaussian Mixed Model with EM algorithm. I cannot find ...
1
vote
1answer
70 views

How to use Bayes' Theorem to detect an event in a noisy signal

I'm trying to use Bayes' Theorem to solve a question that's come up in work, but I don't know if I've done it correctly, because the result seems a bit strange. The problem involves a stochastic ...
1
vote
1answer
3k views

The number of parameters in Gaussian mixture model

I have D-dimensional data with K components. How many parameters if I use a model with full covariance matrices? and How many if I use diaogonal covariance matrices?
1
vote
1answer
607 views

How to do batch learning for Gaussian Mixture Models?

I have a huge dataset of features on which I want to fit a Gaussian Mixture Model using standard expectation maximization, as it is implemented by sklearn. Since not all features fit into the memory ...
1
vote
1answer
336 views

Bimodal univariate distributions are always indicative of a mixture of two random variables. Is this correct? [duplicate]

Say I see a bimodal distribution like this (with the domain, or random variable, $Z$): Does that instantly mean that I am seeing not a distribution of one independent random variable $Z$, but ...
1
vote
1answer
96 views

What does 'vector-valued' mean?

What is the difference of a feature vector and a 'vector-valued observation' as described here? The term 'vector-valued' is used in the following context: "Most state-of-the-art [Automatic Speech ...
1
vote
1answer
158 views

Mixture Model Distributions

I wonder, if there could be a Pareto Mixture Model, just like the Gaussian Mixture Model (GMM). How am I supposed to build a Pareto Mixture Model (PMM)?
1
vote
1answer
117 views

I want to show a local optimum in my paper, how do I generate the data for it?

I'm writing a paper where I am explaining the problems of local optimum in my clustering algorithm. While clustering, in my data I would at times get local optimums. But I've tried and I cannot ...
1
vote
1answer
565 views

My MCMC do not overlap : Mixturemodel with JAGS and R

I fitted a JAGS model and I have those results : My questions are: Why do my chains not overlap, and how can I fix that? I used the following method: My model is a mixture Gaussian model of two ...
1
vote
1answer
223 views

About variance mixture models and probability distributions

I was wondering if anyone knows a good resource to learn about variance mixture models ? My interest is in particular the normal variance mean mixture. I know what they mean with their definition of $...
1
vote
1answer
4k views

Plotting a gaussian mixture with pdf values >> 1 in MATLAB

From a given dataset X, I learn a 7-component gaussian mixture model using matlab's gmdistribution.fit: ...
1
vote
1answer
45 views

Can we use a mixture of normal distributions while optimising likelihood?

Let's assume that we generate some values by a mixture of two Gaussians. Now we want to find the parameters of the two Gaussians by likelihood maximisation. One good expect that the optimisation will ...
1
vote
1answer
172 views

Latent variable in Gaussian Mixture Model

Whenever I look up material pertaining to Gaussian Mixture Models, it always mentions latent variable $z$, where $z \in \{1, ..., K\}$ and is one-hot encoded. I completely understand the objective of ...
1
vote
2answers
187 views

Meaning of Gaussian mixture model parameters

I came across this question from a tutorial: Suppose we have observations $x_1$ , $x_2$ , $\ldots$, $x_n$ of a continuous r.v. $X$ known to be drawn from a “mixture” of $k$ Gaussian distributions. ...
1
vote
1answer
500 views

Why use EM algorithm instead of just plain old ML for mixture model?

Let's say I have some [multivariate] data and want to fit a GMM to it. So I have $P_x=\sum_{i=1}^{n}\alpha_i{N(x;\theta_i)}$, where $x$ is an observation from the data, $\theta_i$ is the mean and ...
1
vote
1answer
1k views

Assessing Gaussian mixture distribution by cross validation

I have a 10 dimensional random vector that I'm modelling with GMMs. I want to estimate the best number of mixtures ($K$) for my data via the following method: Divide the data to train (90%) and test (...
1
vote
2answers
963 views

Manually generate random sample in Gaussian mixture model

I want to generate (manually) a random sample in the Gaussian mixture model: $$f_{\theta}(x) = \sum_{k = 1}^{K}\pi_k f_{\mathcal N(\mu_k, \sigma^2_k)}(x)$$ Here is my work: ...
1
vote
1answer
2k views

Derivation of M-step in EM algorithm for mixture of Gaussians

I am trying to derive the parameter estimation equations for the M-step of the expectation maximization (EM) algorithm for a mixture Gaussians when all Gaussians share the same covariance matrix $\...
1
vote
1answer
1k views

How to know if my Gaussian mixture model has enough training data?

A somewhat soft question - I'm training a Gaussian mixture model (with the EM algorithm) on data of size $N$ ($N$ is typically between 4 and 64). How much samples do I need? obviously it depends on ...
1
vote
1answer
152 views

How to project data onto a model (specifically, GMM)?

I'm using data to train a Gaussian mixture model (GMM). I then take a sample and would like to see its projection on the GMM 'space'. I can think of an optimization problem such as this: consider $y$ ...
1
vote
1answer
1k views

How do I “split” Gaussian mixture components when training EM/GMM based classifier?

In order to improve performance of my Gaussian Mixture Model based classifier, I was recommended to start with a single multivariate Gaussian, estimate its parameters, and "split" it into two mixtures,...
1
vote
2answers
208 views

Mixture Model with dependant observations

I am trying to model a process in which each datapoint is generated sequentially, so the current observation depends on the last one. Some example data could look like, ...
1
vote
2answers
306 views

Finding out if your data belongs to normal distribution

Is there a way to find out if your data belongs to one or more (mixture) normal distributions? I probably could calculate what is the standard deviation of my data, but I'm not sure what else to do ...
1
vote
2answers
3k views

Testing for Unimodality or Bimodality Data Using MATLAB

I am trying to figure out what I did wrong or what I could do to get accurate results. I have n vectors of data, and I am trying to decide whether each dataset is unimodal or bimodal. I assumed that ...
1
vote
1answer
47 views

What is a mixing process?

What does this mean? Asset prices follow a mixture of normal distributions with a mixing process dependent on the unobservable information arrival process.
1
vote
1answer
43 views

Using AIC/BIC within cross-validation for likelihood based loss functions

For a course I am teaching, I am having my students fit a Gaussian mixture model using MLEs via the EM algorithm to a bivariate dataset. I have asked the students to use use cross-validation to choose ...
1
vote
1answer
1k views

Understanding the log-likelihood (score) in scikit-learn GMM

I have been training a GMM (Gaussian Mixture, clustering / unsupervised) on two version of the same dataset: one training with all its features and one training after a PCA truncated to its 2 first ...
1
vote
1answer
19 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
1
vote
1answer
152 views

Conditional distribution in this Gaussian Mixture Model

Say I observe $N$ observations $\{x_1, \dots, x_N\}$ from a $k$ component Gaussian Mixture model, with $k > 0$ known and is such that each $x_i|\boldsymbol{\pi}, \boldsymbol{\mu} \sim \sum_{j=1}^{k}...
1
vote
1answer
100 views

Reducibility between Gaussian Mixture Models and Gaussian Processes

I am studying gaussian processes and I have already discrete amount of knowledge in gaussian mixture models. I am here to undersrtand if with a gaussian process you can fit a gaussian mixture model. ...
1
vote
1answer
129 views

Closed form ML estimation of GMM with known class assignments

In Andrew Ng's CS229 notes, Gaussian mixture model and its likelihood function are given as follows: \begin{eqnarray} z^{(i)} \sim \textrm{Multinomial}(\phi)\\ \phi_j \geq 0\\ \sum_{j=1}^k \phi_j = 1\...
1
vote
1answer
108 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
1
vote
1answer
167 views

Gaussian mixture models with constrained mixing proportions

I am fitting a Gaussian mixture model to multivariate data and my application suggests constraining the mixing proportions to lie in a pre-determined sub-space. I am curious if such an approach has ...
1
vote
1answer
826 views

General conditional distributions for multivariate Gaussian mixtures

My question is similar to this one but considers a more general situation. Suppose that $ \vec{x} = (x_1, \dots, x_d) $ and let $$ p(\vec{x}) = \sum_{k=1}^{n} \pi_k \mathcal{N}(\vec{x} | \mu_k, \...
1
vote
1answer
255 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
1
vote
1answer
241 views

Conditional mean for mixture of multivariate normal distributions

If x = (x_1,x_2,...,x_n) is a vector whose components have a distribution that is a finite mixture of multivariate normals, is the expected value of x_1 still a linear function of the other components,...
1
vote
1answer
574 views

Estimating truncation point in Gaussian mixture

I have data modeled as a mixture of two Gaussian distributions. The data is "clipped" i.e., there is data only for values greater than a threshold $t$, even though it is feasible for data to exist in ...
1
vote
1answer
33 views

Mixture modelling of data with measurement uncertainty

I have a dataset that consists of a population radiometric ages (300>n>600). A dataset can have ages can range on the order of billions of years. Each age measurement has an associated uncertainty ...
1
vote
1answer
323 views

Gradient Ascent for Mixture of Gaussians

Beginner here, apologies if this is something very simple. I am trying to do a gradient ascent to estimate means for a Mixture of Gaussian model. I am using $(x-µ)/σ^3(2π)^{(1/2)} * e^{-((x-µ)/σ)^2)/2}...
1
vote
1answer
216 views

Generate data points for a Gaussian with drawing probability

I am trying to solve this question: Generate 500 data points drawn from each of 3 (three) Gaussians: $N_1(1, 0.1)$, $N_2(1.5., 0.1)$ and $N_3(2, 0.2)$ whose drawing probability on each iteration ...
1
vote
1answer
748 views

How can I find mean and covariance after EM iteration on GMM algorithmm?

I have a dataset divided in 2 class(lets call x1,x2) but I don't know their mean and covariance. For each class I looked their graph and made a guess about their sub-classes, then run an EM(...
1
vote
1answer
604 views

Understanding hidden markov model, and how it is applied in speech recognition

I have for some some time tried to understand how this hidden markov model (hmm) works, and have found a lot of tutorials/papers on it which make use of the same examples/principles of explaining the ...
1
vote
1answer
150 views

Learn parameters for truncated Gaussian

I would like to learn the parameters for a truncated gaussian like this one. I'm using this formula for the probability density $f(x | \mu, \sigma^2) = \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \...
1
vote
1answer
38 views

what is the difference in training and testing for Gaussian and Mixture of Gaussians

what is the difference in training and testing between the Gaussian and Mixture of Gaussians? Are they the same except one is unimodal and one is multimodal?
1
vote
1answer
457 views

Decomposition of multimodal distributions

I have decomposed a multimodal distribution into the constituent single distributions for for further analysis. I have spent some time researching various approaches and I have not found one that that ...
1
vote
1answer
219 views

QDA vs EM with Gaussian likelihoods

QDA (quadratic discriminant analysis) assumes that the K different classes are generated by K different multivariate Gaussians, each with potentially different mean vector and covariance matrix. If ...
1
vote
1answer
528 views

Understanding the E step of EM for GMM

I'm reading this chapter about EM (9.3.1) of the book "Pattern Recognition and Machine Learning". I understand the basic EM algorithm for GMM, but I'm having some problems understanding the ...
1
vote
1answer
747 views

Is the posterior distribution on means in a Bayesian Gaussian mixture model with symmetric priors Gaussian?

I am reading through a document on learning Gaussian mixture models in Infer.NET. They assume the data is generated from 2 Gaussians where the prior distribution on means is Gaussian and the prior ...