Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0
votes
1answer
209 views

JAGS mixture models with exogenous regressors

This is my first post,I hope this is the right forum for such a question and I formulate it correctly. I am working with a time series data set where the response y seems to follow a mixture of two ...
0
votes
1answer
20 views

How many parameters are present in a (general) discrete mixture of five normal distributions?

What is the minimal amount of parameters that can be retained in a particular context?
18
votes
2answers
4k views

Why is optimizing a mixture of Gaussian directly computationally hard?

Consider the log likelihood of a mixture of Gaussians: $$l(S_n; \theta) = \sum^n_{t=1}\log f(x^{(t)}|\theta) = \sum^n_{t=1}\log\left\{\sum^k_{i=1}p_i f(x^{(t)}|\mu^{(i)}, \sigma^2_i)\right\}$$ I was ...
0
votes
0answers
35 views

Determining Link for GLM

I am having a great deal of difficulty understanding how to use the Generalized linear model for my data set. The response variable of interest is hatch success of sea turtles, which is a %. The ...
2
votes
1answer
79 views

Conditional Probability - Mixture Model

I know that the likelihood in a p-dimensional Gaussian mixture model is given by $$ p(s|\theta) = \sum_{b_1 = 0}^1\cdots\sum_{b_p = 0}^1\left[ \prod_{i=1}^pw^{1-b_i}(1-w)^{1-b_i}\right]\phi_p(s|\mu(b,...
1
vote
1answer
228 views

How does scikit-learn calculate a data point's probability of belong to normal distribution?

In GMM calculating, we need to calculate the probability of data point $X$ belong to the $kth$ Gaussian Distribution $\mathcal{N}(X_n|\mu_k,\Sigma_k)$. I have read How to calculate the probability of ...
1
vote
1answer
20 views

What does the y parameter in .fit() of scikit-learn's Gaussian Mixture Model do?

From my understanding, Gaussian Mixture Models are an unsupervised method and can perform clustering similar to k-means. In the scitkit-learn implementation of GMM, the .fit() function (and ....
1
vote
0answers
14 views

How to fit mixture of gaussians with identical mean?

Say I have data generated by a mixture of gaussians whose components have the same mean, but very different covariances, like the one generated by this code: ...
0
votes
0answers
10 views

Probabilistic Models, what do they infer?

As per my understanding, Mixture Models such as GMM, Probabilistic Models such as Variation Autoencoder, they explain the latent space behind the features. But how they turn from learning latent space ...
2
votes
1answer
49 views

Distribution of Distributions | percent point function

I have the following normal distribution (Primary Distribution): Mean = 7 Sigma = 0.5 and a list of other normal distributions (Secondary Distributions): (python list of tupels, each tupel ...
2
votes
1answer
74 views

Figueiredo and Jain's Gaussian mixture EM convergence criterion

I have implemented and been playing around Figueiredo & Jain 's trainer in this paper http://www.lx.it.pt/~mtf/IEEE_TPAMI_2002.pdf for Gaussian mixture. Fig. 2 in the paper depicts the full ...
0
votes
1answer
25 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
0
votes
1answer
233 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
0
votes
0answers
39 views

Gibbs Sampler for GMM

In Rasmussen's paper it is introduced a Gibbs sampler to make inference about a standard Gaussian Mixture Model. To simplify, assume the 1-d case with basic hierarchical structure, that is: $x_i|...
0
votes
2answers
31 views

Convergence of EM algorithm

I am aware that EM eventually converges. However, I still have some confusions regarding this property: 1: As far as I am aware, HMM, Gaussian mixture model and MCMC can converge and all of them use ...
0
votes
0answers
28 views

what is wrong with my Gaussian Mixture density estimation fitting (Python)?

I have a data set (1D) link: dataset, which has values ranging from 21,000 to 8,000,000. When i plot histogram of the log values, i can see there are two peaks, roughly. I tried to fit Gaussian ...
1
vote
0answers
31 views

Closed form for Finite Gaussian Mixture Model when weights are known and prior variance can be 0

Suppose I have a normal likelihood $x|\theta \sim N(\theta, \sigma^2_{known})$ where the variance is known and a mixture prior $\theta \sim p * N(\mu_1, \sigma^2_1) + (1-p) * N(\mu_2, \sigma^2_2)$, ...
6
votes
2answers
842 views

Gaussian Mixture Model - Model selection using the held-out likelihood?

I am trying to understand how to select the number of components in a Gaussian Mixture Model (GMM). Most presentations mention the use of criteria such as AIC and BIC. But if we simply follow model ...
2
votes
1answer
395 views

GMM EM algorithm complexity per iteration

I was fitting GMM clusters with diagonal covariance on my data using EM with $n$ (=5e6) points, each having $m$ (=160) ...
0
votes
2answers
40 views

What is the interpretation of the weights in the GMM?

GMM is $p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the interpretation of the weights here? Do ...
8
votes
1answer
1k views

Simulate from a truncated mixture normal distribution

I want to simulate a sample from a mixture normal distribution such that $$p\times\mathcal{N}(\mu_1,\sigma_1^2) + (1-p)\times\mathcal{N}(\mu_2,\sigma_2^2) $$ is restricted to the interval $[0,1]$ ...
0
votes
2answers
79 views

Suggest a model for this dataset

I have a time series data set (the Old Faithful geyser data available here: http://www.gatsby.ucl.ac.uk/teaching/courses/ml1-2012/geyser.txt). Plotting the eruption duration on the x axis and the ...
2
votes
2answers
50 views

Interpreting mixture of Gaussians (Variational Inference)

I've recently stated reading about mixture models and variational inference in this excellent paper, but I'm having troubles dissecting the models described, and have a couple of questions. Please see ...
1
vote
0answers
19 views

Why mixture model with Gibbs sampling works?

I just have a question about why Gibbs sampling can correctly estimate parameters with random initial value? That is to say,we can sample z by: \begin{align} p(z_i=k \,|\, \cdot) &\...
0
votes
0answers
12 views

How to deduct the complete likelihood of mixture normal in EM algorithm

We have the well known complete likelihood of mixture normal in EM algorithm: Here $Z$ is a random variable that it has probability $\pi_k$ to choose k-th normal variable $X_k:N(\mu_k,\sigma_k).$ We ...
3
votes
2answers
237 views

Universal Approximation Capabilities of Mixture Models

I am looking for two reference incl. proofs showing 1) that a discrete Mixture of Gaussians can asymptotically approximate any (well behaved) continuous density 2) that a discrete Mixture of ...
2
votes
2answers
35 views

Can Gaussian Mixture Model Clustering tell me something about the distribution of my data?

I have 10,000 vectors originating from 5 separate classes (2,000 each). I use Gaussian Mixture Model clustering (in Python) to cluster the 10,000 vectors, telling the algorithm to cluster the data ...
1
vote
1answer
201 views

What is the assumption on the distribution of data in gaussian mixture models?

I am reading about Gaussian mixture models from this slide https://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf However, I am super confused at the very first line. It says: We ...
1
vote
2answers
20 views

Why is Matlab's cluster method able to accept only two inputs? What does it mean when it does? Ex: clusterX = cluster(gmfit,X); [closed]

Matlab's cluster method documentation says cluster takes in 3 arguments: T = cluster(Z,'Cutoff',C) https://www.mathworks.com/help/stats/cluster.html But line 56 inside the cluster function seems to ...
1
vote
2answers
165 views

PCA for probability vectors

Is there a procedure equivalent to principal component analysis (PCA) for probability vectors? I have an n-by-m array where every column sums to one, and all entries are positive. PCA works in ...
0
votes
1answer
216 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
0
votes
0answers
6 views

Decompose 2D matrices into mixture of Gaussians

I have a 2D array that roughly represents a probability distribution in the 2D space. That is, all values in this 2D matrix sum up to 1. I want to decompose this 2D matrix into a sum of Gaussians. ...
2
votes
1answer
231 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
2
votes
1answer
410 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian (...
0
votes
1answer
531 views

Why did log-likelihood decrease in EM in this step-by-step example?

I'm using Expectation Maximization algorithm to determine the parameters of Gaussian distributions in a mixture. To get a better understanding of the algorithm, I executed it manually step by step on ...
4
votes
0answers
39 views

Implementation of EM algorithm confusion

Here EM algorithm manually implemented, there's a question of the implementation in R of the EM algorithm for 2 mixed gaussians. The answer has a supposedly correct implementation. However, don't the ...
1
vote
0answers
20 views

How do I properly scale the covariance matrix in a weighted Gaussian mixture model for new samples?

I am trying to implement the method for computing a Gaussian mixture model from samples with known weights as detailed in section III of: EM Algorithms for Weighted-Data Clustering with Application ...
4
votes
1answer
64 views

Dirichlet process mixture modelling for a Gaussian likelihood

Let $\mathcal{Y} = (\mathbf{y}_1, \dots, \mathbf{y}_N)$ be data observed, such that each $\mathbf{y}_i \in \mathbb{R}^2$. Now conditional on unobserved cluster centres (means) $\mathcal{X} = (\mathbf{...
0
votes
1answer
248 views

Confirmatory latent variable cluster analysis with mplus

I would like to do a confirmatory latent class cluster analyses (finite mixtures) with a continuous and several categorical variables. I know how I can constrain binary variables (such as Cluster A ...
0
votes
0answers
8 views

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture Problem description I am trying to normalize multivariate time series data. The ...
0
votes
1answer
16 views

Expected Misclustering rate

I am reading this paper on minimax clustering error rates on high-dimensional Gaussian mixtures. The authors define a metric for expected misclustering rate as follows: For a two-component ...
1
vote
1answer
547 views

Anomaly detection on 1D data with multiple gaussian distributions

My core problem is to set a cutoff to my one dimension data between normal with abnormal. I think this is a 'anomaly detection' problem. My Data My data is one dimension, consists with below: (...
0
votes
0answers
40 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
0
votes
0answers
29 views

Confidence regions after fitting a 2 parameter gaussian mixture model?

Suppose I have a gaussian mixture model with 2 parameters $(u,v)$ and 2 parts. The model is $P({x_i}|u,v)=uN(x_i|\mu_1^{i} = x_i^2/v,\sigma_1^{i}) + (1-u)N(x_i|\mu_2^{i} = x_i^3/2v^2,\sigma_2^i)$. ...
0
votes
0answers
22 views

Training Hidden Markov model with GMM, nan appears after some iterations

Problem During the training process of my continuous observation sequence data using HMM with GMM mixtures, the cost function reduces gradually and it becomes NaN ...
0
votes
0answers
40 views

Maximum likelihood convergence in mixture gaussian

Suppose there are two datasets $D_{1}$ and $D_{2}$ with same structure, which means the cluster and cluster proportion is the same. The only difference between them is that the size of $D_{1}$ is $n_{...
0
votes
0answers
19 views

How is jaccard similarity used to find the similarity between Bootstrap samples when measuring stability of EM?

Im reading the answer on "how to determine number of clusters in EM algorithm". How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results? One of the ...
0
votes
1answer
74 views

Inverse distribution of gaussian mixture

In one of the papers I've encountered, the authors propose a copula function $$ c(u_1, \ldots, u_d; \Theta) = \frac{\psi(y_1, \ldots, y_d; \Theta)}{\prod_{j=1}^{d}\psi_j (y_j)}$$ where $\psi(y_1, \...
0
votes
0answers
13 views

Biased viterbi training result

I try to use GMM-HMM model to infer the topic of sentences in a short paragraph. While instead of using normal Baum-Welch optimization, I use viterbi training as follows. I use average of word ...