Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
589
questions
0
votes
0
answers
106
views
Conjugate prior bayesian inference on multivariate GMM
I am trying to understand how the posterior looks like when running Bayesian inference on a multivariate Gaussian-mixture model.
$p(\mathbf{x}) \propto \sum_{i=1}^M w_iN(\mathbf{x}|\mu_i,\Sigma_i)$.
...
- 1
0
votes
0
answers
19
views
Is there a formula for the optimal weights for a Gaussian mixture model when the components are considered "correct"?
For simplicity consider two models predicting $Y=Z_0 + \sigma Z_1$, where $Z_i$ are independent $\mathcal{N}(0, 1)$ and each model only take one of the $Z_i$ as a factor. So we have $Y|Z_0$ and $Y|Z_1$...
- 240
1
vote
1
answer
206
views
Assumption of Gaussian mixture model [closed]
I am still confused when I read about the Gaussian Mixture Model and how does it work.
One thing that I do not understand is that "GMM assumes the data is a mixture of Gaussian or
i.i.d (...
- 173
0
votes
0
answers
13
views
Are the subsets of Mixture Models necessarily parametrized?
Im just learning about mixture models and they are described as a "mix of parametric and non parametric models".
My question is how a non-parametric subset would look like? How would e.g. EM-...
- 1
1
vote
1
answer
125
views
Conditional distribution of the weight of a mixture gaussian with data augmentation using gibbs sampling
This question is relate to Differenciate between two distributions using gibbs sampling .
For $t=1,\,\dots,\,n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl}
\...
- 803
0
votes
1
answer
54
views
Differenciate between two distributions using gibbs sampling [closed]
This question is relate to the post :
" Conditional distribution for Gibbs sampling for Gaussian mixture " but is a little bit different. My objective is to know why the algorithm (which is ...
- 803
0
votes
0
answers
43
views
Interpretation of test set negative log likelihood in neural density estimation applications
I have seen people splitting a dataset into a train and test sets and learning the parameters of a mixture density network using the negative log likelihood cost function on the train set and ...
user297721
1
vote
1
answer
424
views
Kernel design for Gaussian processes with multiple inputs
How can I design a kernel function when there are multiple input variables and their degree of influence on the covariance with the target variable is different from each other?
For example, if the ...
- 155
0
votes
0
answers
53
views
Mixture Density Network on Unbalanced Dataset
I want to use a mixture density network to make predictions with NLL loss (which is working) but my dataset is unbalanced so my network tends to make estimations that look like the distribution of ...
- 11
4
votes
1
answer
152
views
Gaussian distribution with poisson variance
Let's $r_t\sim \mathcal{N}(0,V_t)$ where $V_t = (1+m)^{J_t}$, $m\geq0$ and $J_t\sim \mathcal{P}oi(\lambda)$. How can we compute the distribution of $r_t$ when the parameters are $m$ and $\lambda$. In ...
- 803
0
votes
0
answers
225
views
What is truncated gaussian mixture model?
I am interested in the Gaussian mixture model. I read about it and I think I am good with it. However, found that there is something called truncated Gaussian mixture model, which I do not understand. ...
- 1,160
1
vote
2
answers
315
views
Decision boundary between two Gaussians of unequal variance
This question is concerning a similar problem as mentioned in this question. The only difference is that in my case the variances are unequal.
To recap, consider a two class scenario. At the decision ...
- 121
1
vote
1
answer
32
views
Can we "reject" that a distribution is a finite mixture of normals?
Consider a one-dimensional distribution function $f(x)$. Suppose this distribution has all the nice properties, such as continuity, smoothness, etc. We observe $f(x)$.
Suppose that we believe that $f(...
- 403
2
votes
0
answers
99
views
What is the "lower bound average gain" metric used in GMM stopping criterion used in Scikit learn?
In Scikit Learn's GMM class, it says that GMM training algorithm stops according to the "lower bound average gain"
https://scikit-learn.org/stable/modules/generated/sklearn.mixture....
- 1,141
0
votes
1
answer
540
views
Covariance in multivariate Gaussian
In a single dimension Gaussian, the variance $\sigma$ denotes the expected value of the squared deviation from the mean $\mu$.
I am trying to understand why in the multivariate case of modeling ...
- 85
0
votes
1
answer
326
views
Gaussian Mixture Model fails on a simple distribution with a fixed number of components
I'm trying to fit a 2-component 2D Gaussian Mixture Model to some data. I know that there are only two components. The distribution can be seen below in the left plot:
The brain can effortlessly pick ...
- 3,174
2
votes
1
answer
66
views
Is this equivalent to a Gaussian mixture model?
Similar to how you can derive the normal distribution as the distribution where the probability near a point exponentially decays with the square of the number of standard deviations you are away from ...
- 249
1
vote
0
answers
31
views
What is the pdf of the square of the product of two correlated normal distributions?
Let $x$ and $y$ denote a bivariate normal random vector with zero means, unit variances and correlation coefficient $\rho$. Then, the pdf of $z=xy$ is known to be
\begin{equation}
f(z) = \frac{1}{\pi\...
- 414
1
vote
1
answer
176
views
Applying outlier adjustment using student's t distribution in a state-space model
I'm exploring performing outlier adjustment in a state-space model by using student's $t$ distribution. The gist of the problem is formulated as follows:
$$ \begin{align*}
y_t^* &= u_t + o_t - o_{...
- 61
4
votes
2
answers
85
views
Why does it appear impossible to fit Gaussians to arbitrary probability density functions $p$?
I want to fit a Gaussian $q$ to a pdf $p$ by minimizing the energy $E = -\int q(x) \log p(x) dx$. This should result in a "delta function" Gaussian with $\sigma \rightarrow 0$ and $\mu \...
- 147
3
votes
2
answers
117
views
Sum of random variables that follow a finite normal mixture distribution
Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as
$f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
- 1,193
6
votes
1
answer
220
views
A proof that the median is a nonlinear statistical functional
This question is with reference to the top answer (by @StephanKolassa) to this question. Let $F$ and $G$ be CDFs and define $$H(x)=aF(x)+(1-a)G(x)$$ with $a\in [0, 1]$. Now suppose $F$ and $G$ are ...
- 603
2
votes
0
answers
31
views
Mixture regression
I am wondering how to analyze and interpret something like the data shown in the figure below (the color is the log-density of the data points, which are a few hundred thousands in number).
My ...
- 1,660
3
votes
1
answer
43
views
Difference of independent random variables that is not unimodal
This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
- 526
4
votes
0
answers
78
views
How to build a probability distribution from locations with accuracies?
I have a set of $n$ GPS locations $l_i$ with latitude, longitude in degrees and accuracy in meters, corresponding to $3 σ$, i.e. the probability ≈ 0.997) $(lat_i, lng_i, acc_i)$ or $(lat_i, lng_i, σ_i ...
- 141
1
vote
0
answers
257
views
Clustering with gaussian mixtures: choice of hyperparameters
Question: I am interest in general in understanding how to choose the hyperparameters if we are interested in clustering bivariate vectors assuming a mixture of Gaussian mixture with conjugate Normal-...
- 402
1
vote
1
answer
1k
views
How to evaluate the loss on a Gaussian Mixture Model?
I successfully modeled my data using a Gaussian Mixture Model in scikit-learn but I can't figure out how I should say "how good" the model is by calculating the loss.
My first thought was to ...
- 1,300
2
votes
1
answer
403
views
Scikit learn - GMM log-likelihood. Why use Cholesky's precision matrix instead of covariance matrix?
This is my first post, please let me know if I am not being clear.
I am trying to understand the sklearn.mixture.GaussianMixture.score(X).
As I understand that the ...
- 23
2
votes
1
answer
931
views
Understanding the log-likelihood calculation of sklearn Gaussian mixture model
I am trying to understand how the Scipy is calculating the score of a sample in the Gaussian Mixture model(log-likelihood).
Below is the equation I got for log-likelihood from the book C.M. Bishop, ...
- 125
1
vote
0
answers
20
views
Finding maximum likelihood solution for mean when data is given which share the same mean but have different variance
I have some 'X sample points say (x1,x2,x3 ...) each of the samples form a Gaussian distribution with mean 'm' and variance v1,v2, ... All the distributions have the same mean but differ in variance. ...
- 11
2
votes
1
answer
60
views
Upgrading weight parameters to random variable in Gaussian mixtures
In a Gaussian mixture model we model a density like:
$p(\mathbf{x}|\pi,\mu,\sigma)=\sum \pi_i N(\mathbf{x}|\mu_i,\sigma_i)$ [1]
where $\pi,\mu$ and $\sigma$ are parameters.
I would like to know if the ...
- 693
0
votes
1
answer
126
views
About the derivation of EM for mixture of Gaussians
I'm reading Andrew Ng's note about Mixtures of Gaussians and the EM algorithm
He writes the likelihood of data as
where random variables $z^{(i)}$'s indicate which of the $k$ Gaussians each $x^{(i)}$ ...
- 973
0
votes
1
answer
267
views
Fitting Gaussian mixture model with constraints (eg. mu1<mu2) in Python
My question is similar to this one, but while the OP there has constrains such as mu1 being <=0 and mu2 being >=0, my constraints are following:
It's a three component mixture model.
mu1 < ...
- 23
0
votes
1
answer
35
views
Mixture model when K=1
Assume that I want to estimate the parameters of a distribution like for example the Gaussian distribution, but I have the code only for the estimation of the parameters of a mixture of Gaussian ...
- 57
1
vote
0
answers
32
views
Likelihood parameters estimation in mixed data type
How to estimate parameters of a Gaussian mixture model with a mix of categorical and continuous data using log-likelihood?
Indeed, I have a set of data consisting of categorical and continuous data. I ...
- 121
0
votes
0
answers
44
views
Closed form posterior for a mixtures of two univariate Gaussians
Giving a univariate Gaussian mixture model $$\pi_1N(x|\mu_1,\sigma_1)+(1-\pi_1)N(x|\mu_2,\sigma_2),$$ are there any priors for $\pi_1$, $\mu_1$, $\sigma_1$, $\mu_2$, $\sigma_2$ which gives a closed ...
1
vote
0
answers
40
views
Do Gaussian Mixture Models monotonically decrease the sum of squared distances when number of clusters increases?
I am comparing the clustering performance of two closely related machine learning methods: K-means and Gaussian Mixture Models (GMM). Part of this research is selecting the best number of clusters K. ...
- 11
2
votes
1
answer
307
views
Gaussian Mixture Model based clustering for unimodal, time series data
Problem:
I have a simulated data set which is comprised of multiple sub-populations (or samples), each sub-population is drawn from, and described by, its own Gaussian distribution (although by chance,...
- 278
0
votes
0
answers
97
views
Cutoff for a poisson-gaussian mixture model
I have count data that is bounded on one side at zero (see image). It is bimodal and I think it results from two different processes. I would like to fit a poisson distribution around the hump around ...
- 1
2
votes
2
answers
150
views
Estimating weights of known component distributions in a mixture distribution
Given $n$ probability density functions ($p_1$, ..., $p_n$) with known distributions, what are the ways of estimating the weights ($w_1$, ..., $w_n$) of these component distributions given a sample ...
- 23
1
vote
0
answers
354
views
what are the main differences between parametric and non-parametric machine learning algorithms?
I am interested in parametric and non-parametric machine learning algorithms, their advantages and disadvantages and also their main differences regarding computational complexities. In particular I ...
- 11
0
votes
1
answer
28
views
Vatiational inference in GMM
I am learning about VI and am implementing a GMM model for clustering using variational inference. However, my implementation is not fitting the data at all, even when initializing the cluster means ...
- 273
0
votes
0
answers
2k
views
Fitting mixture model of Gaussians and uniform distributions to real data
I have times series of wind direction and velocity. For now, I leave aside the velocity and focus on the distribution of wind directions.
Over there, there is usually three main wind directions, and ...
- 101
4
votes
1
answer
700
views
What is the marginal posterior distribution?
Based on this question: How to build a Bayesian regression model of a response that is a Gaussian mixture
Consider the mixture of normal,
$$y_j\sim (N(0,\sigma_1))^{\pi}(N(0, \sigma_2))^{1-\pi}, j=1,...
user261225
2
votes
0
answers
48
views
Confusion about two Gaussian distributions
From here, it says that, linear combination of two Gaussian distribution, are always Gaussians.
However, Let 𝑋 be standard normal and 𝜀=±1 with probability 1/2 each, independently of 𝑋. Let 𝑌=𝜀𝑋...
- 181
2
votes
0
answers
210
views
How does maximising ELBO for a Gaussian mixture model fit the model to data?
I am following along in Bishop's Pattern Recognition and ML chapters 9 and 10, and I understand that the EM algorithm works by iteratively updating model parameters using equations derived from ...
3
votes
1
answer
231
views
Bayesian mixture model joint posterior
I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
- 966
1
vote
0
answers
64
views
What distribution best describes multiple, sequential normal distributions: What is the sum of more than two normal distributions?
I am curious as to what describes the following distribution. If we were to record some data which are all from a normal distribution, but the standard-deviation changes for blocks of points recorded. ...
- 278
4
votes
1
answer
95
views
Given two normal populations,, classifying a given data point
I have two normal populations S1 and S2, where S1 ~ N (μ1, σ1) and S2 ~ N (μ2, σ2) respectively. The populations are independent of each other and a data point X has to be either from S1 or from S2. ...
- 65
1
vote
1
answer
398
views
Derive cauchy distribution as a scale mixture of normal distributions
I doing Bayesian modelling these days. I found that cauchy distribution can be written as a scale mixture of normal based on following source. Link
So I started to derive this. Somehow, I am not ...
- 821