Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
556
questions
0
votes
0answers
21 views
Variational inference using Gaussian mixture prior
A variational inference provides a way of learning the posterior probability distribution of latent variables of a joint distribution $p(x_{1:T},z_{1:T}, s_{1:T})$ where an observation $x_t$ which is ...
0
votes
1answer
29 views
Gaussian Mixture Model fails on a simple distribution with a fixed number of components
I'm trying to fit a 2-component 2D Gaussian Mixture Model to some data. I know that there are only two components. The distribution can be seen below in the left plot:
The brain can effortlessly pick ...
2
votes
1answer
38 views
Is this equivalent to a Gaussian mixture model?
Similar to how you can derive the normal distribution as the distribution where the probability near a point exponentially decays with the square of the number of standard deviations you are away from ...
0
votes
0answers
18 views
Update of sums of Gasussian Mixture Model
This question is motivated by this post. Then the log multivariate normal density looks like this:
\begin{equation}
\implies
\log(1)+
\frac{p}{2}\log(2\pi)+
\frac{1}{2}\log(|\Sigma^X_j+\...
0
votes
0answers
9 views
Cross-validation for training an SVM classifier on fisher vectors
I have an image classification problem with three classes, each with about 650 training images. Currently, I am using various feature extractors (SURF, HOG, LBP...) to extract features from the images....
1
vote
0answers
16 views
What is the pdf of the square of the product of two correlated normal distributions?
Let $x$ and $y$ denote a bivariate normal random vector with zero means, unit variances and correlation coefficient $\rho$. Then, the pdf of $z=xy$ is known to be
\begin{equation}
f(z) = \frac{1}{\pi\...
0
votes
0answers
9 views
Finding the “center” of many GMMs
Suppose I'm given many GMMs. All have $K$ components. My goal is to find a GMM with $K$ components that can best represent the given GMMs. It is like finding the center of many points but a point here ...
0
votes
1answer
14 views
Entropy of gaussian mixture
Does the entropy of a gaussian mixture depend on its means? It is not the case for a single Gaussian and when the components of the mixture are far spread out, we can approximate the entropy by a ...
1
vote
1answer
81 views
Applying outlier adjustment using student's t distribution in a state-space model
I'm exploring performing outlier adjustment in a state-space model by using student's $t$ distribution. The gist of the problem is formulated as follows:
$$ \begin{align*}
y_t^* &= u_t + o_t - o_{...
0
votes
0answers
12 views
Machine learning models cannot fit, unless gaussian mixture probabilities are used
I was working on the scikit-learn-london dataset (on kaggle) and I was trying to do a binary classification task. The dataset is shortly described below:
40 features (all numerical, not categorical)
...
3
votes
2answers
71 views
Why does it appear impossible to fit Gaussians to arbitrary probability density functions $p$?
I want to fit a Gaussian $q$ to a pdf $p$ by minimizing the energy $E = -\int q(x) \log p(x) dx$. This should result in a "delta function" Gaussian with $\sigma \rightarrow 0$ and $\mu \...
3
votes
2answers
34 views
Sum of random variables that follow a finite normal mixture distribution
Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as
$f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
5
votes
1answer
150 views
A proof that the median is a nonlinear statistical functional
This question is with reference to the top answer (by @StephanKolassa) to this question. Let $F$ and $G$ be CDFs and define $$H(x)=aF(x)+(1-a)G(x)$$ with $a\in [0, 1]$. Now suppose $F$ and $G$ are ...
0
votes
0answers
5 views
K-means to Uncover Finite GMM Parameters in Population Case
Given a finite Gaussian mixture model with the number of distributions known, will k-means reveal the true mean parameters of each Gaussian in the infinite population case? I assume generally not, but ...
0
votes
0answers
10 views
Gaussian Mixture Implementation and Optical Recognition of Handwritten Digits Dataset
Trying to implement Gaussian Mixture model implementation in python using the Optical Recognition of Handwritten Digits Data Set which consists of 10 training folds each of size $\left[100x64\right]$, ...
2
votes
0answers
17 views
Mixture regression
I am wondering how to analyze and interpret something like the data shown in the figure below (the color is the log-density of the data points, which are a few hundred thousands in number).
My ...
3
votes
1answer
31 views
Difference of independent random variables that is not unimodal
This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
4
votes
0answers
69 views
How to build a probability distribution from locations with accuracies?
I have a set of $n$ GPS locations $l_i$ with latitude, longitude in degrees and accuracy in meters, corresponding to $3 σ$, i.e. the probability ≈ 0.997) $(lat_i, lng_i, acc_i)$ or $(lat_i, lng_i, σ_i ...
1
vote
0answers
31 views
Clustering with gaussian mixtures: choice of hyperparameters
Question: I am interest in general in understanding how to choose the hyperparameters if we are interested in clustering bivariate vectors assuming a mixture of Gaussian mixture with conjugate Normal-...
1
vote
1answer
46 views
How to evaluate the loss on a Gaussian Mixture Model?
I successfully modeled my data using a Gaussian Mixture Model in scikit-learn but I can't figure out how I should say "how good" the model is by calculating the loss.
My first thought was to ...
0
votes
0answers
6 views
Partial derivative of composite function of functional data
I would like to find the partial derivative of $f(y)$ with respect to c where $y$ follows multivariate normal/Gaussian density $N(x(t),\sigma^2I_n)$ i.e.
$f(y)=(2\pi)^{-n/2}|\sigma^2I_n|^{-1/2}exp[-1/...
0
votes
0answers
12 views
Find the difference in treatment and control groups during segments of time series data
I have time-series data for a control group and treatment group. I would like to measure the change in current flowing through a cell membrane before and after drug treatment. Each cell is treated ...
2
votes
1answer
102 views
Scikit learn - GMM log-likelihood. Why use Cholesky's precision matrix instead of covariance matrix?
This is my first post, please let me know if I am not being clear.
I am trying to understand the sklearn.mixture.GaussianMixture.score(X).
As I understand that the ...
1
vote
1answer
106 views
Understanding the log-likelihood calculation of sklearn Gaussian mixture model
I am trying to understand how the Scipy is calculating the score of a sample in the Gaussian Mixture model(log-likelihood).
Below is the equation I got for log-likelihood from the book C.M. Bishop, ...
1
vote
0answers
19 views
Finding maximum likelihood solution for mean when data is given which share the same mean but have different variance
I have some 'X sample points say (x1,x2,x3 ...) each of the samples form a Gaussian distribution with mean 'm' and variance v1,v2, ... All the distributions have the same mean but differ in variance. ...
0
votes
0answers
18 views
Major discrepancy of latent variable in the Gaussian Mixture Model/Expectation and Maximization literature
I have read a couple of references on the interpretation of a latent variable in the GMM/EM literature and I found a massive discrepancy between the authors so much so I now have no idea how GMM/EM ...
1
vote
1answer
34 views
Upgrading weight parameters to random variable in Gaussian mixtures
In a Gaussian mixture model we model a density like:
$p(\mathbf{x}|\pi,\mu,\sigma)=\sum \pi_i N(\mathbf{x}|\mu_i,\sigma_i)$ [1]
where $\pi,\mu$ and $\sigma$ are parameters.
I would like to know if the ...
0
votes
1answer
89 views
About the derivation of EM for mixture of Gaussians
I'm reading Andrew Ng's note about Mixtures of Gaussians and the EM algorithm
He writes the likelihood of data as
where random variables $z^{(i)}$'s indicate which of the $k$ Gaussians each $x^{(i)}$ ...
0
votes
1answer
62 views
Fitting Gaussian mixture model with constraints (eg. mu1<mu2) in Python
My question is similar to this one, but while the OP there has constrains such as mu1 being <=0 and mu2 being >=0, my constraints are following:
It's a three component mixture model.
mu1 < ...
0
votes
0answers
15 views
Expectation-Maximization (EM) algorithm with known means
I am trying to fit a mixture of 1D Gaussian distributions to some data. Can the EM algorithm be used in the case of known mean (all the mean values are equal to zero) and fit only the variances of the ...
0
votes
1answer
32 views
Mixture model when K=1
Assume that I want to estimate the parameters of a distribution like for example the Gaussian distribution, but I have the code only for the estimation of the parameters of a mixture of Gaussian ...
1
vote
0answers
24 views
Likelihood parameters estimation in mixed data type
How to estimate parameters of a Gaussian mixture model with a mix of categorical and continuous data using log-likelihood?
Indeed, I have a set of data consisting of categorical and continuous data. I ...
0
votes
0answers
27 views
Closed form posterior for a mixtures of two univariate Gaussians
Giving a univariate Gaussian mixture model $$\pi_1N(x|\mu_1,\sigma_1)+(1-\pi_1)N(x|\mu_2,\sigma_2),$$ are there any priors for $\pi_1$, $\mu_1$, $\sigma_1$, $\mu_2$, $\sigma_2$ which gives a closed ...
1
vote
0answers
26 views
Do Gaussian Mixture Models monotonically decrease the sum of squared distances when number of clusters increases?
I am comparing the clustering performance of two closely related machine learning methods: K-means and Gaussian Mixture Models (GMM). Part of this research is selecting the best number of clusters K. ...
1
vote
1answer
80 views
Gaussian Mixture Model based clustering for unimodal, time series data
Problem:
I have a simulated data set which is comprised of multiple sub-populations (or samples), each sub-population is drawn from, and described by, its own Gaussian distribution (although by chance,...
0
votes
0answers
17 views
Why can't the parameters of Gaussian mixture models be estimated in closed form?
I'm new to ML and I'm reading up on density estimation with Gaussian Mixture Models. I read that the parameters of Gaussian mixture models cannot be estimated in closed form, but I'm not sure why. Why ...
0
votes
0answers
65 views
Gaussian Mixture Models and distance matrix
I have a (euclidean) distance matrix and I want to perform GMM clustering.
I read in another post (gaussian mixture model - approximate a matrix) that I could apply MDS or PCA to this matrix and use ...
0
votes
0answers
37 views
Cutoff for a poisson-gaussian mixture model
I have count data that is bounded on one side at zero (see image). It is bimodal and I think it results from two different processes. I would like to fit a poisson distribution around the hump around ...
2
votes
2answers
67 views
Estimating weights of known component distributions in a mixture distribution
Given $n$ probability density functions ($p_1$, ..., $p_n$) with known distributions, what are the ways of estimating the weights ($w_1$, ..., $w_n$) of these component distributions given a sample ...
1
vote
0answers
103 views
what are the main differences between parametric and non-parametric machine learning algorithms?
I am interested in parametric and non-parametric machine learning algorithms, their advantages and disadvantages and also their main differences regarding computational complexities. In particular I ...
0
votes
0answers
15 views
How to parameterize variational Dirichlet distribution
I am learning about variational inference and am implementing a couple of things from scratch. I am trying to build a Gaussian mixture model where the prior on the mixture component selection is a ...
0
votes
1answer
15 views
Vatiational inference in GMM
I am learning about VI and am implementing a GMM model for clustering using variational inference. However, my implementation is not fitting the data at all, even when initializing the cluster means ...
0
votes
0answers
16 views
Gaussian mixture model for image labelling task
I'm trying to solve an image labelling task by using Gaussian Mixture Models. The total number of classes in my dataset is 9, each representing a different variety of vegetable (Class1, Class2, Class3)...
0
votes
0answers
20 views
GMM for nonlinear mean
In conventional GMM, observations $\mathbf{X} = \left\lbrace \mathbf{x}_1,\mathbf{x}_2,\ldots\right\rbrace$ are draw from a distribution
$$
\mathbf{x}_n \sim \sum_{k=1}^{K}\pi_k\mathcal{N}\left( \...
0
votes
0answers
374 views
Fitting mixture model of Gaussians and uniform distributions to real data
I have times series of wind direction and velocity. For now, I leave aside the velocity and focus on the distribution of wind directions.
Over there, there is usually three main wind directions, and ...
4
votes
1answer
300 views
What is the marginal posterior distribution?
Based on this question: How to build a Bayesian regression model of a response that is a Gaussian mixture
Consider the mixture of normal,
$$y_j\sim (N(0,\sigma_1))^{\pi}(N(0, \sigma_2))^{1-\pi}, j=1,...
2
votes
0answers
22 views
Confusion about two Gaussian distributions
From here, it says that, linear combination of two Gaussian distribution, are always Gaussians.
However, Let 𝑋 be standard normal and 𝜀=±1 with probability 1/2 each, independently of 𝑋. Let 𝑌=𝜀𝑋...
0
votes
0answers
30 views
What is the nature of $r_i^t$?
Using Expectation Maximization (EM) algorithm, I want to vary the number of clusters used according to $K = [2,4, ... 50]$ for a normal distribution initialized randomly (...
2
votes
0answers
92 views
How does maximising ELBO for a Gaussian mixture model fit the model to data?
I am following along in Bishop's Pattern Recognition and ML chapters 9 and 10, and I understand that the EM algorithm works by iteratively updating model parameters using equations derived from ...
0
votes
0answers
11 views
Estimating the means in an equal mixture of two Gaussians with known variance 1
Consider data from an equal mixture of two Gaussians with variance 1:
$X \sim 1/2 \mathcal{N}(\mu_1, 1) + 1/2 \mathcal{N}(\mu_2, 1)$.
The means can be estimated with an EM algorithm, but is there ...
