Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
569
questions
1
vote
0answers
11 views
What determines performance in recoverying K in Gaussian Mixture Model?
My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm.
For simplicity, let's consider the case in ...
1
vote
1answer
14 views
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Recently, I came across the use of Gaussian Mixture Distributions being used to model the response ...
0
votes
0answers
48 views
+50
How to calculate the covariance matrix in (Xu and Knight 2010)?
Setup
I'm reading (Xu and Knight 2010), which is a paper about estimating finite Gaussian mixture models using the CECF (Continuous Empirical Characteristic Function) method. The basic idea is to ...
4
votes
1answer
64 views
K Means as a special case of GMM (using EM Algorithm)
I am looking for a tutorial/gentle introduction (preferably with mathematics/proofs) on K-means as a special case of Gaussian Mixture Model using the EM Algorithm.
I have found this: https://www....
0
votes
0answers
25 views
Compute log-likelihood in Bernoulli Gaussian Mixture
I'm working on this exercise about Gaussian Mixtures:
Here's part of the solution:
I don't understand how they came up with this equation for the log-likelihood (red arrow).
From what I know, the ...
0
votes
0answers
20 views
Help students understand Gaussian Kernel in mean shift segmentation
I am creating a image mean shift segmentation algorithm for a class.
So far what I've done is basically
Iterate through each point/pixel on the image and with a given bandwidth(radius) calculate the ...
1
vote
2answers
101 views
Are linear combinations (i.e. "sums") of gaussian distributions also gaussian?
I have always assumed that this is fact : but are there any mathematical theorems that state: finite sums of gaussian distributions are gaussian themselves?
In the above picture, does p(x) have a ...
0
votes
0answers
15 views
Conjugate prior bayesian inference on multivariate GMM
I am trying to understand how the posterior looks like when running Bayesian inference on a multivariate Gaussian-mixture model.
$p(\mathbf{x}) \propto \sum_{i=1}^M w_iN(\mathbf{x}|\mu_i,\Sigma_i)$.
...
0
votes
0answers
17 views
computing the mutual information of a mixture of Gaussians
How can one compute the mutual information of Gaussian mixture distribution
\begin{equation}
\mathrm{I}(X;C|Y)=H[C|Y]-H(C|X,Y)=-\int\int\sum_{C}P(X,C,Y)\log\frac{P(X,C|Y)}{P(C|Y)P(X|Y)}\mathrm{d}X\...
0
votes
0answers
8 views
Is there a formula for the optimal weights for a Gaussian mixture model when the components are considered "correct"?
For simplicity consider two models predicting $Y=Z_0 + \sigma Z_1$, where $Z_i$ are independent $\mathcal{N}(0, 1)$ and each model only take one of the $Z_i$ as a factor. So we have $Y|Z_0$ and $Y|Z_1$...
1
vote
1answer
58 views
Assumption of Gaussian mixture model [closed]
I am still confused when I read about the Gaussian Mixture Model and how does it work.
One thing that I do not understand is that "GMM assumes the data is a mixture of Gaussian or
i.i.d (...
0
votes
0answers
12 views
Are the subsets of Mixture Models necessarily parametrized?
Im just learning about mixture models and they are described as a "mix of parametric and non parametric models".
My question is how a non-parametric subset would look like? How would e.g. EM-...
0
votes
1answer
77 views
Conditional distribution of the weight of a mixture gaussian with data augmentation using gibbs sampling
This question is relate to Differenciate between two distributions using gibbs sampling .
For $t=1,\,\dots,\,n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl}
\...
0
votes
1answer
49 views
Differenciate between two distributions using gibbs sampling [closed]
This question is relate to the post :
" Conditional distribution for Gibbs sampling for Gaussian mixture " but is a little bit different. My objective is to know why the algorithm (which is ...
0
votes
0answers
18 views
Interpretation of test set negative log likelihood in neural density estimation applications
I have seen people splitting a dataset into a train and test sets and learning the parameters of a mixture density network using the negative log likelihood cost function on the train set and ...
0
votes
1answer
61 views
Kernel design for Gaussian processes with multiple inputs
How can I design a kernel function when there are multiple input variables and their degree of influence on the covariance with the target variable is different from each other?
For example, if the ...
0
votes
0answers
32 views
Mixture Density Network on Unbalanced Dataset
I want to use a mixture density network to make predictions with NLL loss (which is working) but my dataset is unbalanced so my network tends to make estimations that look like the distribution of ...
3
votes
1answer
123 views
Gaussian distribution with poisson variance
Let's $r_t\sim \mathcal{N}(0,V_t)$ where $V_t = (1+m)^{J_t}$, $m\geq0$ and $J_t\sim \mathcal{P}oi(\lambda)$. How can we compute the distribution of $r_t$ when the parameters are $m$ and $\lambda$. In ...
0
votes
0answers
8 views
Serie calculation - Normal poisson distribution
I would like to compute that serie if it is possible :
$\sum_{j\geq 0}\frac{(\lambda m^{-1/2})^j}{j!} \exp{\left(-B\,m^{-j}\right)}$ where $B \geq 0,\, \lambda > 0,\, m>1$.
I know it converge as ...
1
vote
0answers
23 views
Tied Bayesian Mixture of Gaussians
I am bit confused when it comes to modelling a Bayesian Gaussian mixture model that assumes a shared covariance/precision matrix for all Gaussian components.
I followed the derivation in Bishop and ...
0
votes
0answers
59 views
Number of free parameters of a hidden Markov model with emission prob. as Gaussian mixture model?
If there are $K$ components (or HMM states)
and there is $D$ dimensional GMM
is the number of free parameters is:
HMM:
$K(K-1)$ for transition matrix
$K-1$ for prior
GMM:
$D(D+1)K/2$ for ...
0
votes
0answers
32 views
What is truncated gaussian mixture model?
I am interested in the Gaussian mixture model. I read about it and I think I am good with it. However, found that there is something called truncated Gaussian mixture model, which I do not understand. ...
2
votes
2answers
118 views
Decision boundary between two Gaussians of unequal variance
This question is concerning a similar problem as mentioned in this question. The only difference is that in my case the variances are unequal.
To recap, consider a two class scenario. At the decision ...
1
vote
1answer
26 views
Can we "reject" that a distribution is a finite mixture of normals?
Consider a one-dimensional distribution function $f(x)$. Suppose this distribution has all the nice properties, such as continuity, smoothness, etc. We observe $f(x)$.
Suppose that we believe that $f(...
0
votes
0answers
13 views
How does the MLE of Gaussian mixture model give a square covariance matrix?
I'm reading these lecture notes. There's this equation:
For calculating the covariance matrix. I don't really understand how this results in a square covariance matrix. Cause for example the ...
2
votes
0answers
43 views
What is the "lower bound average gain" metric used in GMM stopping criterion used in Scikit learn?
In Scikit Learn's GMM class, it says that GMM training algorithm stops according to the "lower bound average gain"
https://scikit-learn.org/stable/modules/generated/sklearn.mixture....
0
votes
1answer
41 views
Covariance in multivariate Gaussian
In a single dimension Gaussian, the variance $\sigma$ denotes the expected value of the squared deviation from the mean $\mu$.
I am trying to understand why in the multivariate case of modeling ...
0
votes
1answer
97 views
Gaussian Mixture Model fails on a simple distribution with a fixed number of components
I'm trying to fit a 2-component 2D Gaussian Mixture Model to some data. I know that there are only two components. The distribution can be seen below in the left plot:
The brain can effortlessly pick ...
2
votes
1answer
48 views
Is this equivalent to a Gaussian mixture model?
Similar to how you can derive the normal distribution as the distribution where the probability near a point exponentially decays with the square of the number of standard deviations you are away from ...
0
votes
0answers
19 views
Update of sums of Gasussian Mixture Model
This question is motivated by this post. Then the log multivariate normal density looks like this:
\begin{equation}
\implies
\log(1)+
\frac{p}{2}\log(2\pi)+
\frac{1}{2}\log(|\Sigma^X_j+\...
0
votes
0answers
11 views
Cross-validation for training an SVM classifier on fisher vectors
I have an image classification problem with three classes, each with about 650 training images. Currently, I am using various feature extractors (SURF, HOG, LBP...) to extract features from the images....
1
vote
0answers
25 views
What is the pdf of the square of the product of two correlated normal distributions?
Let $x$ and $y$ denote a bivariate normal random vector with zero means, unit variances and correlation coefficient $\rho$. Then, the pdf of $z=xy$ is known to be
\begin{equation}
f(z) = \frac{1}{\pi\...
0
votes
0answers
13 views
Finding the "center" of many GMMs
Suppose I'm given many GMMs. All have $K$ components. My goal is to find a GMM with $K$ components that can best represent the given GMMs. It is like finding the center of many points but a point here ...
0
votes
1answer
50 views
Entropy of gaussian mixture
Does the entropy of a gaussian mixture depend on its means? It is not the case for a single Gaussian and when the components of the mixture are far spread out, we can approximate the entropy by a ...
1
vote
1answer
123 views
Applying outlier adjustment using student's t distribution in a state-space model
I'm exploring performing outlier adjustment in a state-space model by using student's $t$ distribution. The gist of the problem is formulated as follows:
$$ \begin{align*}
y_t^* &= u_t + o_t - o_{...
0
votes
0answers
24 views
Machine learning models cannot fit, unless gaussian mixture probabilities are used
I was working on the scikit-learn-london dataset (on kaggle) and I was trying to do a binary classification task. The dataset is shortly described below:
40 features (all numerical, not categorical)
...
4
votes
2answers
78 views
Why does it appear impossible to fit Gaussians to arbitrary probability density functions $p$?
I want to fit a Gaussian $q$ to a pdf $p$ by minimizing the energy $E = -\int q(x) \log p(x) dx$. This should result in a "delta function" Gaussian with $\sigma \rightarrow 0$ and $\mu \...
3
votes
2answers
75 views
Sum of random variables that follow a finite normal mixture distribution
Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as
$f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
6
votes
1answer
171 views
A proof that the median is a nonlinear statistical functional
This question is with reference to the top answer (by @StephanKolassa) to this question. Let $F$ and $G$ be CDFs and define $$H(x)=aF(x)+(1-a)G(x)$$ with $a\in [0, 1]$. Now suppose $F$ and $G$ are ...
0
votes
0answers
5 views
K-means to Uncover Finite GMM Parameters in Population Case
Given a finite Gaussian mixture model with the number of distributions known, will k-means reveal the true mean parameters of each Gaussian in the infinite population case? I assume generally not, but ...
0
votes
0answers
19 views
Gaussian Mixture Implementation and Optical Recognition of Handwritten Digits Dataset
Trying to implement Gaussian Mixture model implementation in python using the Optical Recognition of Handwritten Digits Data Set which consists of 10 training folds each of size $\left[100x64\right]$, ...
2
votes
0answers
22 views
Mixture regression
I am wondering how to analyze and interpret something like the data shown in the figure below (the color is the log-density of the data points, which are a few hundred thousands in number).
My ...
3
votes
1answer
37 views
Difference of independent random variables that is not unimodal
This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
4
votes
0answers
74 views
How to build a probability distribution from locations with accuracies?
I have a set of $n$ GPS locations $l_i$ with latitude, longitude in degrees and accuracy in meters, corresponding to $3 σ$, i.e. the probability ≈ 0.997) $(lat_i, lng_i, acc_i)$ or $(lat_i, lng_i, σ_i ...
1
vote
0answers
83 views
Clustering with gaussian mixtures: choice of hyperparameters
Question: I am interest in general in understanding how to choose the hyperparameters if we are interested in clustering bivariate vectors assuming a mixture of Gaussian mixture with conjugate Normal-...
1
vote
1answer
443 views
How to evaluate the loss on a Gaussian Mixture Model?
I successfully modeled my data using a Gaussian Mixture Model in scikit-learn but I can't figure out how I should say "how good" the model is by calculating the loss.
My first thought was to ...
0
votes
0answers
9 views
Partial derivative of composite function of functional data
I would like to find the partial derivative of $f(y)$ with respect to c where $y$ follows multivariate normal/Gaussian density $N(x(t),\sigma^2I_n)$ i.e.
$f(y)=(2\pi)^{-n/2}|\sigma^2I_n|^{-1/2}exp[-1/...
0
votes
0answers
21 views
Find the difference in treatment and control groups during segments of time series data
I have time-series data for a control group and treatment group. I would like to measure the change in current flowing through a cell membrane before and after drug treatment. Each cell is treated ...
2
votes
1answer
196 views
Scikit learn - GMM log-likelihood. Why use Cholesky's precision matrix instead of covariance matrix?
This is my first post, please let me know if I am not being clear.
I am trying to understand the sklearn.mixture.GaussianMixture.score(X).
As I understand that the ...
2
votes
1answer
310 views
Understanding the log-likelihood calculation of sklearn Gaussian mixture model
I am trying to understand how the Scipy is calculating the score of a sample in the Gaussian Mixture model(log-likelihood).
Below is the equation I got for log-likelihood from the book C.M. Bishop, ...
