Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Under What Conditions Does a Gaussian Mixture Model (GMM) Have Maximum Entropy?

Introduction I'm delving into Gaussian Mixture Models (GMMs) within unsupervised learning frameworks and am particularly interested in their statistical properties, with a focus on entropy. Entropy ...
Alireza Ghazavi's user avatar
0 votes
0 answers
12 views

Expected Variance of EM Estimator in GMM with Respect to Observations

Title: Variance of EM Estimator in GMM with Respect to Observations Body: I'm estimating a parameter S from observations X and <...
Alireza Ghazavi's user avatar
0 votes
0 answers
8 views

Multivariate mixture models with INLA: Relating CAR random effect to the MVN covariance matrix

For a research project I am trying to implement a multivariate mixture model on areal data using Integrated Nested Laplace Approximation (INLA). Let $y_{i,d}$ be a data point, where $i = 1, 2, \cdots, ...
BasMts's user avatar
  • 23
0 votes
0 answers
12 views

Online mixture inference; better alternatives than windowed EM?

I have an online Gaussian mixture estimation problem that I would appreciate some input on. To be more precise, I have a stream of scalar observations $x_1, x_2, \dotsc$ arriving over time which are ...
ummg's user avatar
  • 145
0 votes
0 answers
22 views

Can I assume that this is a GMM?

I'm trying to find the MLE for the parameters of the following distribution: $$f(x) = a \ \mathcal{N}(\mu_a, 1) + \beta \ \mathcal{N}(\mu_\beta, 1)$$ Taking the log likelihood of this complicates ...
John Katsantas's user avatar
0 votes
0 answers
10 views

X and Y are correlated, errors in both X and Y but error variances unknown; How to predict X|Y or Y|X? Deming, bivariate gaussian ellipses, other?

Seeking relationships between two variable, both with random gaussian errors; ratio of error variances is unknown, no correlation of errors in X and Y, but another unknown variable Z (unmeasured) may ...
David's user avatar
  • 1
0 votes
0 answers
12 views

Computational issue for finding the JS divergence

I had two datasets with 144 points in 2 dimension then I used he sklearn library to fit the GMM and that fits well , I checked for the BIC values while fitting the model and chooses the model with low ...
tiger's user avatar
  • 1
0 votes
0 answers
21 views

Question about Gaussian Mixture model [duplicate]

suppose I have two parameters: $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and suppose I mapped them to PDF using the gaussian mixture ...
Andyale's user avatar
  • 133
0 votes
0 answers
29 views

Derivation of EM algorithm for Gaussian mixture

I am going through Expectation Maximization (EM) algorithm derivation for Gaussian Mixture models. I understand it except for a small detail. So, the general idea of EM is to maximize the expectation ...
baronett's user avatar
2 votes
0 answers
40 views

Derive ELBO for Mixture of Gaussian

I am working through "Variational Inference: A Review for Statisticians" by Blei et al. (see https://arxiv.org/abs/1601.00670) and they illustrate Variational Inference using a Bayesian ...
PSLS's user avatar
  • 41
0 votes
0 answers
70 views

Understanding of Gamma distribution as precision prior in Bayesian inference for Gaussian

Christopher M. Bishop in his book "Pattern Recognition and Machine Learning" nicely explains where does Student t-distribution $St(x|\mu,\lambda,\upsilon)$ originate into. In Chapter 2, it ...
baronett's user avatar
0 votes
1 answer
45 views

Identify outliers in testing data based on trained Gaussian mixture model

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
0 answers
81 views

Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
1 answer
63 views

Gaussian Mixture Model with Minkowski distance

Gaussian Mixture Models assume Mahalanobis distance (essentially L2). Is it possible to use Lp distance in a GMM? Intuitively, in 1-space, distance is clear. In 2-space, the relation between the two ...
olivarb's user avatar
  • 111
0 votes
0 answers
14 views

Are combinations of variables a problem in multinomial mixture models?

I am quite new to cluster analysis and I am currently using Gaussian-multinomial mixture models. I have a sample size of 10000 people with some continuous variables and about 14 categorical variables (...
Lander's user avatar
  • 3
2 votes
2 answers
141 views

MAP estimation for a Gaussian mixture using EM. Concerns with the covariance update formula

I am implementing the EM algorithm for a Gaussian mixture model with prior; that is, I am using the EM algorithm to find the MAP estimate, rather than the ML estimate. As briefly discussed in section ...
ummg's user avatar
  • 145
1 vote
0 answers
26 views

Linear Discriminant Analysis with unlabeled data

In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following: From the mixture formulation [that is, ...
Sergio's user avatar
  • 336
1 vote
1 answer
120 views

Generate marginally dependent (with predetermined covariance) but conditionally independent data from a Mixture of Gaussians

Suppose you have three variables $y\in\{0,1\}$ and $x_1\in\mathbb{R}$ and $x_2\in\mathbb{R}$. I want to produce data with the following generative process which corresponds to a Mixture of Gaussians (...
Sergio's user avatar
  • 336
0 votes
0 answers
15 views

Mixture Model: Data Consist of Continuous and Binary Features

I have a features like below id x1 x2 x3 x4 x5 id1 0.4 1.4 5.6 1 0 id2 -0.01 0.5 -3.4 0 1 where x1, x2, x3 are continuous features and x4 and x5 are binary. The goal is to find $k$ clusters using ...
AnonymousJ's user avatar
0 votes
0 answers
15 views

Supervised classifier for nested interval data and ordinal classes

I'm having trouble formalizing the following classification problem: Let $x_i$ denote univariate (scalar), continuous, real data points Let $y_i \in \mathbb{N}$ be their corresponding labels Classes ...
themodelguy's user avatar
0 votes
1 answer
59 views

Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model

I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
Kyungmin's user avatar
0 votes
1 answer
38 views

Assumptions and setting for bayesian mixture model (for RJMCMC)

I want to understand about Bayesian mixture model discussed in RJMCMC paper (Richardson and Green, 1997) (https://academic.oup.com/jrsssb/article/59/4/731/7083042) I also posted similar question ...
Kyungmin's user avatar
0 votes
0 answers
55 views

EM algorithm get new parameters by optimizing the Q function (lower bound of likelihood function) or optimizing the likelihood function

We know that in the EM (Expectation-Maximization) algorithm, the E-step determines the $Q$ function by calculating expectations, which is a lower bound of the likelihood function. In the M-step, by ...
yi ren's user avatar
  • 1
2 votes
1 answer
237 views

Why gaussian mixture model is sensitive to cluster size?

I did an experiment where I generated 3 well separated clusters with a different multivariate gaussian distribution for each cluster. One of the clusters contains 1000 points and the other two ...
striker22's user avatar
  • 323
0 votes
2 answers
67 views

Flexmix maxima are not where they are expected to be

For my dataset I have plotted the density with ggplot. As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus. ...
Lukas's user avatar
  • 111
0 votes
0 answers
23 views

Decomposing into Gaussian components using Bhattacharya from topFishR

I am working with fishery data. I have a data vector called SFL that contains the sizes of the fish caught. Here is some sample data: ...
sermomon's user avatar
  • 113
0 votes
1 answer
53 views

Confidence interval for unsymmetrical Gaussian Mixure Model PDFs?

Let Y be a vector of observations. A Gaussian Mixure Model (GMM) is fit to the dataset. The distribution can appear unsymmetrical, with different thickness of tails in both sides. What is the best way ...
Bahar's user avatar
  • 1
0 votes
1 answer
82 views

Fitting a Gaussian Mixture Model with known share of noise/outliers

A Gaussian Mixture model is fitted by the Expectation-Maximization algorithm. This fairly simple iterative algorithm consists of two steps and the initialization. Initialization (for ...
Klops's user avatar
  • 156
1 vote
1 answer
146 views

Variance of a mixture of normally distributed random variables [duplicate]

Setup Suppose I have two normally distributed random variables, $s_\mathrm{prev}$ (with mean $\mu_\mathrm{prev}$ and variance $\sigma^2_\mathrm{prev}$) and $s_\mathrm{curr}$ (with mean $\mu_\mathrm{...
abcd's user avatar
  • 113
0 votes
0 answers
15 views

Requisites in data in order to use Gaussian Mixture Models

I would like to know if I want to use GMM if the data has to follow a Gaussian distribution even if I preprocess the data and normalize it. Is it enough to normalize the data or has it have to pass ...
Jesus M.'s user avatar
0 votes
0 answers
150 views

Bayesian Gaussian mixture - is my prior correct?

I'd like to sample from the Bayesian Posterior of a Gaussian mixture model, but I am not sure about the correct Bayesian formulation of the latter. Is the following correct? I consider the 1-...
reloh100's user avatar
1 vote
0 answers
56 views

Could one use mixtures of Gaussians to turn MCMC posterior samples into a new prior?

Theoretically in Bayesian inference one could use one experiment's posterior as another experiment's prior, such that knowledge of the parameters accumulates from $p(\theta) \rightarrow p(\theta|\...
Durden's user avatar
  • 1,255
0 votes
0 answers
57 views

Differential Entropy of Zero-Mean Gaussian Mixtures

Introduction Consider a univariate circularly symmetric complex Gaussian (CSCG) mixture $Y$ with pdf $$p_Y(y) = \sum_i c_i p_i(y) = \sum_i c_i \frac{\exp(-\lvert y \rvert^2/\sigma_i^2)}{\pi \sigma_i^2}...
SnowzTail's user avatar
0 votes
0 answers
11 views

What assumptions are made about categorical variables in regression models? [duplicate]

This is the formalization for a continuous regression model as I understand it: Assume that your outcome $Y$ is normally distributed. Assume that your predictors, $X_1, X_2, \cdots X_n$ are normally ...
djpassey's user avatar
0 votes
0 answers
15 views

What is the difference between hidden states and mixture components in a continuous Hidden Markov Model?

I'm trying to understand how a continuous Hidden Markov Model works and I am confused by the difference of the hidden states of the discrete latent variables and the mixture components in the emission ...
Alfonso Lava's user avatar
0 votes
0 answers
55 views

Clustering correlated variables in Gaussian Mixture Models

I would like to ask it dose not require to consider the correlation of variables if using Gaussian mixture model for clustering, because covariance has already been considered in the model. So the ...
Xiaose's user avatar
  • 1
0 votes
0 answers
20 views

Similarity of two sets of scatter data

I am currently working on a project where I have a dataset of scatter-points in space. What I am doing is that I fit Gaussian Mixture Models (GMM) on the data set and use the model to sample '...
Kelvin Walenta's user avatar
0 votes
0 answers
34 views

How to sample from a given accessible PDF?

I am learning Gibbs Sampling for GMMs. Particularly, given $\boldsymbol \theta$, I must sample from the latent $\boldsymbol z$ before sampling $\boldsymbol x$. The PDF of $\boldsymbol z$ is given as$$ ...
wd violet's user avatar
  • 635
1 vote
0 answers
269 views

What does mixture refer to in LCMM and HLME in R

I am trying to use the HLME and LCMM functions to fit latent class mixed models to my data. Here are the documentations to both of them: https://www.rdocumentation.org/packages/lcmm/versions/1.8.1.1/...
TYL's user avatar
  • 213
1 vote
1 answer
183 views

Gaussian mixture model probabilities

I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
alle_meije's user avatar
1 vote
0 answers
39 views

How to solve gamma from GMM

GMM refers to the Gaussian mixture model, cf. here. Suppose we have $N$ data points (or observations in Statistics) and $K$ Gaussian models to mix. After going through Maximum Likelihood Estimation, ...
Sam Wong's user avatar
  • 177
5 votes
1 answer
450 views

Mixture of Gaussian is not log-concave

I've encountered the statement: For $p\in(0,1),$ the location mixture of standard univariate normal densities $f(x)=p\phi(x)+(1-p)\phi(x-\mu)$ is log-concave if and only if $\Vert\mu\Vert \leq 2.$ ...
12345's user avatar
  • 213
1 vote
1 answer
43 views

Mixture of Two Normals

Suppose we have a data which consists of two normals, x = rnorm(50,mean=1,sd=2) y = rnorm(50,mean=2,sd=3) z = sample( c(x,y) , size = 100, replace=FALSE ) The goal ...
Nicolas Bourbaki's user avatar
0 votes
0 answers
105 views

Model Comparison within Bayesian Gaussian Mixture Model framework

Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model. $$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
Fiodor1234's user avatar
  • 2,194
7 votes
1 answer
297 views

EM algorithm for Bivariate Normal

Consider a random sample $X_i = (U_i,V_i)$ where $i=1,2,...,n$ from a bivariate normal population with mean $(\mu_1,\mu_2)$ and variances $(\sigma_1 ^2, \sigma_2 ^2)$ and correlation $\rho$. Let's ...
Maths Freak's user avatar
0 votes
1 answer
775 views

Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]

In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
Vincent Granville's user avatar
1 vote
0 answers
26 views

What is the best way to calculate the fit of the mixture distribution to the actual data?

I have fitted the mixture of Gaussians to the natural log of the data. I know that the model is not a very good fit to the data in the tail region, however in the high density region the actual data ...
Karina's user avatar
  • 85
1 vote
1 answer
161 views

Closed form solution of multivariate Gaussian over mixture of multivariate Gaussians

Suppose I have three variables $X_{1}, X_{2}$ and $Y$, where $X_{1}, X_{2}$ are continuous and $Y$ is binary. The conditional distribution of $X_{1}, X_{2}$, given $Y$ is a multivariate Gaussian ...
Sergio's user avatar
  • 336
1 vote
1 answer
76 views

Coefficients of Gaussian mixture

This is in context of Gaussian mixtures $$p(\boldsymbol{x}) = \sum_{k=1}^K \pi_k\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})$$ Bishop mentions on Page-111 Also, the requirement ...
muser's user avatar
  • 135
2 votes
2 answers
703 views

p(x) in Gaussian mixture model

I've recently learned about Gaussian mixture models (GMM) in school. The formula for a GMM goes something like this: $$p(x) = \sum_{k=1}^{K} \pi_i\mathcal{N}(x|\mu_i,\,\sigma^{2}_i)$$ Now, I know from ...
Dude156's user avatar
  • 123

1
2 3 4 5
13