Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
542
questions
1
vote
0answers
16 views
Clustering with gaussian mixtures: choice of hyperparameters
Question: I am interest in general in understanding how to choose the hyperparameters if we are interested in clustering bivariate vectors assuming a mixture of Gaussian mixture with conjugate Normal-...
1
vote
1answer
17 views
How to evaluate the loss on a Gaussian Mixture Model?
I successfully modeled my data using a Gaussian Mixture Model in scikit-learn but I can't figure out how I should say "how good" the model is by calculating the loss.
My first thought was to ...
0
votes
0answers
5 views
Partial derivative of composite function of functional data
I would like to find the partial derivative of $f(y)$ with respect to c where $y$ follows multivariate normal/Gaussian density $N(x(t),\sigma^2I_n)$ i.e.
$f(y)=(2\pi)^{-n/2}|\sigma^2I_n|^{-1/2}exp[-1/...
0
votes
0answers
10 views
Find the difference in treatment and control groups during segments of time series data
I have time-series data for a control group and treatment group. I would like to measure the change in current flowing through a cell membrane before and after drug treatment. Each cell is treated ...
2
votes
1answer
56 views
Scikit learn - GMM log-likelihood. Why use Cholesky's precision matrix instead of covariance matrix?
This is my first post, please let me know if I am not being clear.
I am trying to understand the sklearn.mixture.GaussianMixture.score(X).
As I understand that the ...
1
vote
1answer
49 views
Understanding the log-likelihood calculation of sklearn Gaussian mixture model
I am trying to understand how the Scipy is calculating the score of a sample in the Gaussian Mixture model(log-likelihood).
Below is the equation I got for log-likelihood from the book C.M. Bishop, ...
1
vote
0answers
18 views
Finding maximum likelihood solution for mean when data is given which share the same mean but have different variance
I have some 'X sample points say (x1,x2,x3 ...) each of the samples form a Gaussian distribution with mean 'm' and variance v1,v2, ... All the distributions have the same mean but differ in variance. ...
0
votes
0answers
14 views
Major discrepancy of latent variable in the Gaussian Mixture Model/Expectation and Maximization literature
I have read a couple of references on the interpretation of a latent variable in the GMM/EM literature and I found a massive discrepancy between the authors so much so I now have no idea how GMM/EM ...
1
vote
1answer
32 views
Upgrading weight parameters to random variable in Gaussian mixtures
In a Gaussian mixture model we model a density like:
$p(\mathbf{x}|\pi,\mu,\sigma)=\sum \pi_i N(\mathbf{x}|\mu_i,\sigma_i)$ [1]
where $\pi,\mu$ and $\sigma$ are parameters.
I would like to know if the ...
0
votes
1answer
72 views
About the derivation of EM for mixture of Gaussians
I'm reading Andrew Ng's note about Mixtures of Gaussians and the EM algorithm
He writes the likelihood of data as
where random variables $z^{(i)}$'s indicate which of the $k$ Gaussians each $x^{(i)}$ ...
0
votes
1answer
47 views
Fitting Gaussian mixture model with constraints (eg. mu1<mu2) in Python
My question is similar to this one, but while the OP there has constrains such as mu1 being <=0 and mu2 being >=0, my constraints are following:
It's a three component mixture model.
mu1 < ...
0
votes
0answers
13 views
Expectation-Maximization (EM) algorithm with known means
I am trying to fit a mixture of 1D Gaussian distributions to some data. Can the EM algorithm be used in the case of known mean (all the mean values are equal to zero) and fit only the variances of the ...
0
votes
1answer
31 views
Mixture model when K=1
Assume that I want to estimate the parameters of a distribution like for example the Gaussian distribution, but I have the code only for the estimation of the parameters of a mixture of Gaussian ...
1
vote
0answers
23 views
Likelihood parameters estimation in mixed data type
How to estimate parameters of a Gaussian mixture model with a mix of categorical and continuous data using log-likelihood?
Indeed, I have a set of data consisting of categorical and continuous data. I ...
0
votes
0answers
25 views
Closed form posterior for a mixtures of two univariate Gaussians
Giving a univariate Gaussian mixture model $$\pi_1N(x|\mu_1,\sigma_1)+(1-\pi_1)N(x|\mu_2,\sigma_2),$$ are there any priors for $\pi_1$, $\mu_1$, $\sigma_1$, $\mu_2$, $\sigma_2$ which gives a closed ...
1
vote
0answers
26 views
Do Gaussian Mixture Models monotonically decrease the sum of squared distances when number of clusters increases?
I am comparing the clustering performance of two closely related machine learning methods: K-means and Gaussian Mixture Models (GMM). Part of this research is selecting the best number of clusters K. ...
1
vote
1answer
71 views
Gaussian Mixture Model based clustering for unimodal, time series data
Problem:
I have a simulated data set which is comprised of multiple sub-populations (or samples), each sub-population is drawn from, and described by, its own Gaussian distribution (although by chance,...
0
votes
0answers
14 views
Why can't the parameters of Gaussian mixture models be estimated in closed form?
I'm new to ML and I'm reading up on density estimation with Gaussian Mixture Models. I read that the parameters of Gaussian mixture models cannot be estimated in closed form, but I'm not sure why. Why ...
0
votes
0answers
31 views
Gaussian Mixture Models and distance matrix
I have a (euclidean) distance matrix and I want to perform GMM clustering.
I read in another post (gaussian mixture model - approximate a matrix) that I could apply MDS or PCA to this matrix and use ...
0
votes
0answers
22 views
Cutoff for a poisson-gaussian mixture model
I have count data that is bounded on one side at zero (see image). It is bimodal and I think it results from two different processes. I would like to fit a poisson distribution around the hump around ...
2
votes
2answers
47 views
Estimating weights of known component distributions in a mixture distribution
Given $n$ probability density functions ($p_1$, ..., $p_n$) with known distributions, what are the ways of estimating the weights ($w_1$, ..., $w_n$) of these component distributions given a sample ...
1
vote
0answers
85 views
what are the main differences between parametric and non-parametric machine learning algorithms?
I am interested in parametric and non-parametric machine learning algorithms, their advantages and disadvantages and also their main differences regarding computational complexities. In particular I ...
0
votes
0answers
12 views
How to parameterize variational Dirichlet distribution
I am learning about variational inference and am implementing a couple of things from scratch. I am trying to build a Gaussian mixture model where the prior on the mixture component selection is a ...
0
votes
1answer
14 views
Vatiational inference in GMM
I am learning about VI and am implementing a GMM model for clustering using variational inference. However, my implementation is not fitting the data at all, even when initializing the cluster means ...
0
votes
0answers
15 views
Gaussian mixture model for image labelling task
I'm trying to solve an image labelling task by using Gaussian Mixture Models. The total number of classes in my dataset is 9, each representing a different variety of vegetable (Class1, Class2, Class3)...
0
votes
0answers
18 views
GMM for nonlinear mean
In conventional GMM, observations $\mathbf{X} = \left\lbrace \mathbf{x}_1,\mathbf{x}_2,\ldots\right\rbrace$ are draw from a distribution
$$
\mathbf{x}_n \sim \sum_{k=1}^{K}\pi_k\mathcal{N}\left( \...
0
votes
0answers
167 views
Fitting mixture model of Gaussians and uniform distributions to real data
I have times series of wind direction and velocity. For now, I leave aside the velocity and focus on the distribution of wind directions.
Over there, there is usually three main wind directions, and ...
4
votes
1answer
254 views
What is the marginal posterior distribution?
Based on this question: How to build a Bayesian regression model of a response that is a Gaussian mixture
Consider the mixture of normal,
$$y_j\sim (N(0,\sigma_1))^{\pi}(N(0, \sigma_2))^{1-\pi}, j=1,...
2
votes
0answers
22 views
Confusion about two Gaussian distributions
From here, it says that, linear combination of two Gaussian distribution, are always Gaussians.
However, Let 𝑋 be standard normal and 𝜀=±1 with probability 1/2 each, independently of 𝑋. Let 𝑌=𝜀𝑋...
0
votes
0answers
30 views
What is the nature of $r_i^t$?
Using Expectation Maximization (EM) algorithm, I want to vary the number of clusters used according to $K = [2,4, ... 50]$ for a normal distribution initialized randomly (...
2
votes
0answers
60 views
How does maximising ELBO for a Gaussian mixture model fit the model to data?
I am following along in Bishop's Pattern Recognition and ML chapters 9 and 10, and I understand that the EM algorithm works by iteratively updating model parameters using equations derived from ...
0
votes
0answers
9 views
Estimating the means in an equal mixture of two Gaussians with known variance 1
Consider data from an equal mixture of two Gaussians with variance 1:
$X \sim 1/2 \mathcal{N}(\mu_1, 1) + 1/2 \mathcal{N}(\mu_2, 1)$.
The means can be estimated with an EM algorithm, but is there ...
0
votes
0answers
21 views
gaussian mixture model bic using simulated data
I'm trying to generate data for a Gaussian Simulation model, and then test if mclustBIC (mclust package) can correctly suggest the number of components. There are three methods of generating data: ...
0
votes
0answers
20 views
Finding posterior probabilities on GMM
I was searching for how to properly implement GMM in a Bayesian setting, and I have read in a PyMC3 example the following:
Marginalizing [the categorical labels] out of the model generally leads to ...
4
votes
1answer
80 views
Bayesian mixture model joint posterior
I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
0
votes
0answers
40 views
Why does the smallest eigenvalue of the covariance matrix of a GMM equal the common variance?
I've been reading a paper "Introduction to Tensor Decompositions and Their Applications in Machine Learning". In it, the author describes an algorithm for estimating the means of the ...
1
vote
0answers
30 views
What distribution best describes multiple, sequential normal distributions: What is the sum of more than two normal distributions?
I am curious as to what describes the following distribution. If we were to record some data which are all from a normal distribution, but the standard-deviation changes for blocks of points recorded. ...
4
votes
1answer
66 views
Given two normal populations,, classifying a given data point
I have two normal populations S1 and S2, where S1 ~ N (μ1, σ1) and S2 ~ N (μ2, σ2) respectively. The populations are independent of each other and a data point X has to be either from S1 or from S2. ...
0
votes
1answer
69 views
Derive cauchy distribution as a scale mixture of normal distributions
I doing Bayesian modelling these days. I found that cauchy distribution can be written as a scale mixture of normal based on following source. Link
So I started to derive this. Somehow, I am not ...
0
votes
0answers
18 views
Probabilistic generative models for clustering and classification
I have a question regarding the probabilistic setting of clustering and classification. More specifically regarding Gaussian Mixture Models and probabilistic generative models for classification.
In ...
1
vote
1answer
37 views
Is 0 the unique center for the mixture density of $N(-a,\sigma^2)$ and $N(a,\sigma^2)$, each with weight 0.5? [duplicate]
Suppose $f_{-a}(x)$ is the pdf for $N(-a,\sigma^2)$ and $f_{a}(x)$ is the pdf for $N(a,\sigma^2)$.
Let $f(x)=0.5f_{-a}(x)+0.5f_{a}(x)$ be the mixture density.
Is $c=0$ the unique center for $f(x)$ in ...
11
votes
4answers
2k views
In cluster analysis, how does Gaussian mixture model differ from K Means when we know the clusters are spherical?
I understand how main difference between K-mean and Gaussian mixture model (GMM) is that K-Mean only detects spherical clusters and GMM can adjust its self to elliptic shape cluster. However, how do ...
1
vote
1answer
106 views
How to get number of iterations in EM-algorithm using R mclust gaussian mixture model
I am clustering data using the mclust function from the R mclust package. I am struggling to get the number of iterations the EM ...
1
vote
1answer
193 views
Bayes' theorem in Gaussian mixture model for $p(z_i = k | x_i, \mu_k,\Sigma_k)$
Given the $k$th Gaussian distribution $N \sim (\mu_k, \Sigma_k)$, the probability that $x_i$ generated from this Gaussian $k$ can be found via Bayes' rule $$\begin{align}p(z_i = k | x_i,\mu_k, \...
1
vote
1answer
38 views
Gaussian Mixture Model $p(x_i | z_i = k)$ a likelihood or probability?
In Gaussian Mixture models, the probability of observing the data $x$ given that it was generated from $M$ gaussian models is given by the following equation $$p(x) = \sum_{k=1}^m p(x|z=k)p(z = k)$$
...
0
votes
1answer
118 views
Probability Density Function and Maximum Likelihood Estimation for Multinomial Logistic Regression and GMM
I have some confusion about a few very basic concepts and terminology.
Let's assume we have two models for classification, a multinomial logistic regression (MLR) model and a GMM classifier. I'm not ...
0
votes
0answers
32 views
Derivation of maximum likelihood estimation of Gaussian mixture model
I want to derive the following formula.
The meaning of each expression is as follows
It's easy to solve with Kronecker's delta.
0
votes
0answers
83 views
Confidence intervals for mixture of Gaussian distributions
I have a mixture distribution of 2 Gaussians. Here, the left has a weight of 0.1 and the right has a weight of 0.9. In this example, they have identical $\sigma$, but that may not always be the case....
0
votes
1answer
139 views
Order and interpretation of Gaussian Mixture Model with strong overlap between components
Most examples for Gaussian Mixture Models (GMMs) employ datasets with fairly obvious underlying structure (well-separated clusters). How should one determine the order of a GMM (and interpret the ...
0
votes
0answers
12 views
How to obtain equation for evidence given equation for joint density of latent and observed variables (Variational Inference)
I am working through the following review paper: https://arxiv.org/pdf/1601.00670.pdf
and would like to gain some insight into arriving at the formulae for joint density of latent and observed ...
