Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
620
questions
0
votes
0
answers
21
views
Question about Gaussian Mixture model [duplicate]
suppose I have two parameters:
$(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and suppose I mapped them to PDF using the gaussian mixture ...
- 133
0
votes
0
answers
22
views
Derivation of EM algorithm for Gaussian mixture
I am going through Expectation Maximization (EM) algorithm derivation for Gaussian Mixture models. I understand it except for a small detail.
So, the general idea of EM is to maximize the expectation ...
- 11
2
votes
0
answers
16
views
Derive ELBO for Mixture of Gaussian
I am working through "Variational Inference: A Review for Statisticians" by Blei et al. (see https://arxiv.org/abs/1601.00670) and they illustrate Variational Inference using a Bayesian ...
- 41
0
votes
0
answers
57
views
Understanding of Gamma distribution as precision prior in Bayesian inference for Gaussian
Christopher M. Bishop in his book "Pattern Recognition and Machine Learning" nicely explains where does Student t-distribution $St(x|\mu,\lambda,\upsilon)$ originate into. In Chapter 2, it ...
- 11
0
votes
1
answer
38
views
Identify outliers in testing data based on trained Gaussian mixture model
I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as:
$p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
- 11
1
vote
0
answers
67
views
Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]
I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as:
$p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
- 11
1
vote
1
answer
46
views
Gaussian Mixture Model with Minkowski distance
Gaussian Mixture Models assume Mahalanobis distance (essentially L2). Is it possible to use Lp distance in a GMM?
Intuitively, in 1-space, distance is clear. In 2-space, the relation between the two ...
- 111
0
votes
0
answers
14
views
Are combinations of variables a problem in multinomial mixture models?
I am quite new to cluster analysis and I am currently using Gaussian-multinomial mixture models. I have a sample size of 10000 people with some continuous variables and about 14 categorical variables (...
- 3
2
votes
2
answers
109
views
MAP estimation for a Gaussian mixture using EM. Concerns with the covariance update formula
I am implementing the EM algorithm for a Gaussian mixture model with prior; that is, I am using the EM algorithm to find the MAP estimate, rather than the ML estimate. As briefly discussed in section ...
- 135
1
vote
0
answers
23
views
Linear Discriminant Analysis with unlabeled data
In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following:
From the mixture formulation [that is, ...
- 336
1
vote
1
answer
116
views
Generate marginally dependent (with predetermined covariance) but conditionally independent data from a Mixture of Gaussians
Suppose you have three variables $y\in\{0,1\}$ and $x_1\in\mathbb{R}$ and $x_2\in\mathbb{R}$. I want to produce data with the following generative process which corresponds to a Mixture of Gaussians (...
- 336
0
votes
0
answers
15
views
Mixture Model: Data Consist of Continuous and Binary Features
I have a features like below
id
x1
x2
x3
x4
x5
id1
0.4
1.4
5.6
1
0
id2
-0.01
0.5
-3.4
0
1
where x1, x2, x3 are continuous features and x4 and x5 are binary. The goal is to find $k$ clusters using ...
0
votes
0
answers
15
views
Supervised classifier for nested interval data and ordinal classes
I'm having trouble formalizing the following classification problem:
Let $x_i$ denote univariate (scalar), continuous, real data points
Let $y_i \in \mathbb{N}$ be their corresponding labels
Classes ...
0
votes
1
answer
48
views
Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model
I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
- 45
0
votes
1
answer
37
views
Assumptions and setting for bayesian mixture model (for RJMCMC)
I want to understand about Bayesian mixture model discussed in RJMCMC paper (Richardson and Green, 1997) (https://academic.oup.com/jrsssb/article/59/4/731/7083042)
I also posted similar question ...
- 45
0
votes
0
answers
50
views
EM algorithm get new parameters by optimizing the Q function (lower bound of likelihood function) or optimizing the likelihood function
We know that in the EM (Expectation-Maximization) algorithm, the E-step determines the $Q$ function by calculating expectations, which is a lower bound of the likelihood function. In the M-step, by ...
- 1
2
votes
1
answer
175
views
Why gaussian mixture model is sensitive to cluster size?
I did an experiment where I generated 3 well separated clusters with a different multivariate gaussian distribution for each cluster. One of the clusters contains 1000 points and the other two ...
- 323
0
votes
2
answers
56
views
Flexmix maxima are not where they are expected to be
For my dataset I have plotted the density with ggplot.
As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus.
...
- 111
0
votes
0
answers
20
views
Decomposing into Gaussian components using Bhattacharya from topFishR
I am working with fishery data. I have a data vector called SFL that contains the sizes of the fish caught. Here is some sample data:
...
- 113
0
votes
1
answer
42
views
Confidence interval for unsymmetrical Gaussian Mixure Model PDFs?
Let Y be a vector of observations. A Gaussian Mixure Model (GMM) is fit to the dataset. The distribution can appear unsymmetrical, with different thickness of tails in both sides. What is the best way ...
- 1
0
votes
1
answer
53
views
Fitting a Gaussian Mixture Model with known share of noise/outliers
A Gaussian Mixture model is fitted by the Expectation-Maximization algorithm.
This fairly simple iterative algorithm consists of two steps and the initialization.
Initialization (for ...
- 156
1
vote
1
answer
110
views
Variance of a mixture of normally distributed random variables [duplicate]
Setup
Suppose I have two normally distributed random variables, $s_\mathrm{prev}$ (with mean $\mu_\mathrm{prev}$ and variance $\sigma^2_\mathrm{prev}$) and $s_\mathrm{curr}$ (with mean $\mu_\mathrm{...
- 113
0
votes
0
answers
13
views
Requisites in data in order to use Gaussian Mixture Models
I would like to know if I want to use GMM if the data has to follow a Gaussian distribution even if I preprocess the data and normalize it. Is it enough to normalize the data or has it have to pass ...
- 1
0
votes
0
answers
103
views
Bayesian Gaussian mixture - is my prior correct?
I'd like to sample from the Bayesian Posterior of a Gaussian mixture model, but I am not sure about the correct Bayesian formulation of the latter. Is the following correct?
I consider the 1-...
- 1
1
vote
0
answers
45
views
Could one use mixtures of Gaussians to turn MCMC posterior samples into a new prior?
Theoretically in Bayesian inference one could use one experiment's posterior as another experiment's prior, such that knowledge of the parameters accumulates from $p(\theta) \rightarrow p(\theta|\...
- 955
0
votes
0
answers
47
views
Differential Entropy of Zero-Mean Gaussian Mixtures
Introduction
Consider a univariate circularly symmetric complex Gaussian (CSCG) mixture $Y$ with pdf
$$p_Y(y) = \sum_i c_i p_i(y) = \sum_i c_i \frac{\exp(-\lvert y \rvert^2/\sigma_i^2)}{\pi \sigma_i^2}...
0
votes
0
answers
11
views
What assumptions are made about categorical variables in regression models? [duplicate]
This is the formalization for a continuous regression model as I understand it:
Assume that your outcome $Y$ is normally distributed. Assume that your predictors, $X_1, X_2, \cdots X_n$ are normally ...
- 1
0
votes
0
answers
15
views
What is the difference between hidden states and mixture components in a continuous Hidden Markov Model?
I'm trying to understand how a continuous Hidden Markov Model works and I am confused by the difference of the hidden states of the discrete latent variables and the mixture components in the emission ...
0
votes
0
answers
52
views
Clustering correlated variables in Gaussian Mixture Models
I would like to ask it dose not require to consider the correlation of variables if using Gaussian mixture model for clustering, because covariance has already been considered in the model. So the ...
- 1
0
votes
0
answers
18
views
Similarity of two sets of scatter data
I am currently working on a project where I have a dataset of scatter-points in space. What I am doing is that I fit Gaussian Mixture Models (GMM) on the data set and use the model to sample '...
0
votes
0
answers
33
views
How to sample from a given accessible PDF?
I am learning Gibbs Sampling for GMMs. Particularly, given $\boldsymbol \theta$, I must sample from the latent $\boldsymbol z$ before sampling $\boldsymbol x$.
The PDF of $\boldsymbol z$ is given as$$
...
- 605
1
vote
0
answers
237
views
What does mixture refer to in LCMM and HLME in R
I am trying to use the HLME and LCMM functions to fit latent class mixed models to my data. Here are the documentations to both of them:
https://www.rdocumentation.org/packages/lcmm/versions/1.8.1.1/...
- 203
1
vote
1
answer
164
views
Gaussian mixture model probabilities
I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
- 113
1
vote
0
answers
37
views
How to solve gamma from GMM
GMM refers to the Gaussian mixture model, cf. here.
Suppose we have $N$ data points (or observations in Statistics) and $K$ Gaussian models to mix. After going through Maximum Likelihood Estimation, ...
- 177
5
votes
1
answer
412
views
Mixture of Gaussian is not log-concave
I've encountered the statement:
For $p\in(0,1),$ the location mixture of standard univariate normal densities
$f(x)=p\phi(x)+(1-p)\phi(x-\mu)$ is log-concave if and only if $\Vert\mu\Vert \leq 2.$
...
- 213
0
votes
1
answer
40
views
Mixture of Two Normals
Suppose we have a data which consists of two normals,
x = rnorm(50,mean=1,sd=2)
y = rnorm(50,mean=2,sd=3)
z = sample( c(x,y) , size = 100, replace=FALSE )
The goal ...
- 2,266
0
votes
0
answers
98
views
Model Comparison within Bayesian Gaussian Mixture Model framework
Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model.
$$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
- 2,152
7
votes
1
answer
258
views
EM algorithm for Bivariate Normal
Consider a random sample $X_i = (U_i,V_i)$ where $i=1,2,...,n$ from a bivariate normal population with mean $(\mu_1,\mu_2)$ and variances $(\sigma_1 ^2, \sigma_2 ^2)$ and correlation $\rho$. Let's ...
- 217
0
votes
1
answer
652
views
Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]
In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
1
vote
0
answers
25
views
What is the best way to calculate the fit of the mixture distribution to the actual data?
I have fitted the mixture of Gaussians to the natural log of the data. I know that the model is not a very good fit to the data in the tail region, however in the high density region the actual data ...
- 85
1
vote
1
answer
144
views
Closed form solution of multivariate Gaussian over mixture of multivariate Gaussians
Suppose I have three variables $X_{1}, X_{2}$ and $Y$, where $X_{1}, X_{2}$ are continuous and $Y$ is binary. The conditional distribution of $X_{1}, X_{2}$, given $Y$ is a multivariate Gaussian ...
- 336
1
vote
1
answer
68
views
Coefficients of Gaussian mixture
This is in context of Gaussian mixtures
$$p(\boldsymbol{x}) = \sum_{k=1}^K \pi_k\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})$$
Bishop mentions on Page-111
Also, the requirement ...
- 135
2
votes
2
answers
605
views
p(x) in Gaussian mixture model
I've recently learned about Gaussian mixture models (GMM) in school. The formula for a GMM goes something like this:
$$p(x) = \sum_{k=1}^{K} \pi_i\mathcal{N}(x|\mu_i,\,\sigma^{2}_i)$$
Now, I know from ...
- 123
1
vote
0
answers
17
views
When imputing missing labels Y1 == NaN during training, how do additional target vectors (Y2 != NaN) impact learning Y1==NaN?
I am training a Mixture Density Network (MDN) to map from continuous input vectors X to continuous targets Y [i.e. X -> Y]. There are missing labels on vector Yi, which I impute from the mdn (as ...
- 11
0
votes
1
answer
97
views
Could the likelihood increase monotonically in a misspecified EM algorithm?
I am dealing with the estimation of a Gaussian Hidden Markov Model with conditional distribution given the first-order Markov state $S_t = j,\ j=1,...,J$
$$
Y_t|S_t=j\sim N(0,\sigma^2_j)
$$
where the ...
- 329
0
votes
0
answers
28
views
Metric to check the number of clusters in one dimentional data
I have a set of 1D data as shown below.
I want to find a metric that represents the number of clusters in data.
Is there any suitable metric that matches my criteria?
Example 1D data list:
[68, 3, 3, ...
- 1
0
votes
0
answers
36
views
Is there methods for splitting a Guassian Mixture model into separate unimodal distributions?
I want to apply an algorithm which usually works on unimodal distributions on multimodal distributions, I'm considering doing this by splitting the multimodal distributions into multiple unimodal ...
1
vote
0
answers
14
views
How do I make a table that prints the Mean and its corresponding weight of a Gaussian Mixture model?
I am successfully generating weights and means for a GMM, but I'm trying to get the results to print in a way that shows the corresponding mean for a weight. Here's what I have now:
...
- 11
1
vote
0
answers
107
views
Finding Probability that data is taken from a Gaussian Mixture Model
I'm using scikit's Gaussian Mixture to obtain a 2 component mixture model for some 1-D data. Having obtained a model, I want to test how well a different data set matches this mixture model, or more ...
- 33
2
votes
1
answer
680
views
Calculating weighted covariance matrix of a weighted finite mixture of multivariate normal distributions
I am trying to calculate the weighted covariance matrix for a finite mixture of multivariate normal distributions. I read this post here and this one here, but the first post is focused on uniformly ...
